{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "线性回归：\n",
    "\n",
    "线性回归就是**计算回归系数，通过回归系数线性组合属性预测结果值。**\n",
    "\n",
    "$$f(x)=w^Tx+b$$\n",
    "\n",
    "二分类的线性分类：\n",
    "\n",
    "**只不过在线性回归的基础上条件了大于和小于0的判断。**\n",
    "\n",
    "$$ y= \\begin{cases} 1, & \\text {f(x)$\\geq$0} \\\\ -1, & \\text{f(x)<0} \\end{cases} $$\n",
    "\n",
    "使用线性分类时存在**线性可分和线性不可分。**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "比如：下图为线性可分的\n",
    "\n",
    "![这里写图片描述](https://img-blog.csdnimg.cn/img_convert/eed1dfe978e29447ac13ad86f77f5932.png)\n",
    "\n",
    "当然，也存在着许多线性不可分的情况，例如下图所示\n",
    "\n",
    "![这里写图片描述](https://img-blog.csdnimg.cn/img_convert/8f83806b31b1c992a6b93bc2bb6cb3bd.png)\n",
    "\n",
    "\n",
    "即使是线性可分的，分类边界也不一定是确定的。\n",
    "\n",
    "比如下面的几个分类边界都是可以实现分类的，哪个更好呢。\n",
    "\n",
    "![这里写图片描述](https://img-blog.csdnimg.cn/img_convert/dd5465c00b80f37b097fb66d33bf929b.png)\n",
    "\n",
    "这就需要用到支持向量机SVM算法了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 支持向量机/SVM ##\n",
    "\n",
    "SVM（Support Vector Machines）是分类算法中应用广泛、效果不错的一类。\n",
    "\n",
    "由简至繁SVM可分类为三类：线性可分（linear SVM in linearly separable case）的线性SVM、线性不可分的线性SVM、非线性（nonlinear）SVM。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. 线性可分\n",
    "\n",
    "\n",
    "对于二类分类问题，训练集$T={(x_1,y_1),(x_2,y_2),⋯,(x_N,y_N)}$，其中$x_i$为输入样本向量，类别$y_i∈{-1,1}$，线性SVM通过学习得到分离超平面（hyperplane）:\n",
    "\n",
    "$$w⋅x+b=0$$\n",
    "\n",
    "以及相应的分类决策函数：\n",
    "\n",
    "$$f(x)=sign(w⋅x+b)$$\n",
    "\n",
    "其中$sign$为符号函数，将正数映射为1，负数映射为-1。\n",
    "\n",
    "有如下图所示的分离超平面，哪一个超平面的分类效果更好呢？\n",
    "\n",
    "![这里写图片描述](https://img-blog.csdnimg.cn/img_convert/d49f06e99d682a71fd8b5c43c0d8ba01.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "直观上，超平面B1的分类效果更好一些。将距离分离超平面最近的两个不同类别的样本点称为支持向量（support vector）的，构成了两条平行于分离超平面的长带，二者之间的距离称之为margin。\n",
    "\n",
    "显然，margin更大，则分类正确的确信度更高（与超平面的距离表示分类的确信度，距离越远则分类正确的确信度越高）。\n",
    "\n",
    "通过计算得到（计算过程参考《统计学习方法》115页）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$margin=\\frac{2}{∥w∥}$$\n",
    "\n",
    "从上图中可观察到：margin以外的样本点对于确定分离超平面没有贡献，换句话说，SVM是有很重要的训练样本（支持向量）所确定的。\n",
    "\n",
    "至此，SVM分类问题可描述为在全部分类正确的情况下，最大化$\\frac{2}{∥w∥}$（等价于最小化$0.5∥w∥^2$）；\n",
    "\n",
    "线性分类的约束最优化问题转化为如下的凸优化问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> $$\\min_{w,b}\\frac{1}{2}||w||^2$$ $$s.t.\\quad \\quad           y_i(w\\cdot x_i+b)-1 \\geq 0$$\n",
    "\n",
    "\n",
    "在线性可分情况下，训练数据集的样本点中与分离超平面距离最近的样本点的实例称为支持向量。\n",
    "\n",
    "支持向量满足$$ y_i(w\\cdot x_i+b)-1 =0$$\n",
    "\n",
    "即正样本点满足$$ w\\cdot x_i+b=1$$\n",
    "\n",
    "负样本点满足$$ w\\cdot x_i+b=-1$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. 线性支持向量机\n",
    "\n",
    "\n",
    "线性可分是理想情形，大多数情况下，由于噪声或特异点等各种原因，训练样本是线性不可分的。\n",
    "\n",
    "因此，需要更一般化的学习算法。\n",
    "\n",
    "这时我们就可以通过引入所谓的松弛变量(slack variable)，来允许有些数据点可以处于超平面的错误的一侧。\n",
    "\n",
    "这样我们的优化目标就能保持仍然不变，但是此时我们的约束条件有所改变。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 凸优化问题\n",
    "\n",
    "\n",
    "（a）无约束优化问题，可以写为：\n",
    "\n",
    "$$min\\quad f(x)$$\n",
    "\n",
    "（b）有等式约束的优化问题，可以写为：\n",
    "\n",
    "$$min\\quad f(x)  \\\\\n",
    "s.t. \\quad h_i(x)=0 ,i=0,1,2...n$$\n",
    "\n",
    "（c）有不等式约束的优化问题，可以写为：\n",
    "\n",
    "$$min\\quad f(x)  \\\\\n",
    "s.t. \\quad g_i(x) \\leq 0 ,i=0,1,2...n\\\\\n",
    " h_j(x)=0 ,j=0,1,2...m$$\n",
    "\n",
    "对于第(a)类的优化问题，常常使用的方法就是费马大定理(Fermat)，即使用求取函数f(x)的导数，然后令其为零，可以求得候选最优值，再在这些候选值中验证；\n",
    "\n",
    "如果是凸函数，可以保证是最优解。\n",
    "\n",
    "这也就是我们高中经常使用的求函数的极值的方法。\n",
    "\n",
    "对于第(b)类的优化问题，常常使用的方法就是拉格朗日乘子法（Lagrange Multiplier) ，即把等式约束h_i(x)用一个系数与f(x)写为一个式子，称为拉格朗日函数，而系数称为拉格朗日乘子。\n",
    "\n",
    "通过拉格朗日函数对各个变量求导，令其为零，可以求得候选值集合，然后验证求得最优值。\n",
    "\n",
    "对于第(c)类的优化问题，常常使用的方法就是KKT条件。\n",
    "\n",
    "同样地，我们把所有的等式、不等式约束与f(x)写为一个式子，也叫拉格朗日函数，系数也称拉格朗日乘子，通过一些条件，可以求出最优值的必要条件，这个条件称为KKT条件。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "必要条件和充要条件如果不理解，可以看下面这句话：\n",
    "\n",
    " - 事件M的必要条件就是M可以推出的结论 \n",
    " - 事件M的充分条件就是可以推出M的前提\n",
    "\n",
    "了解到这些，现在让我们再看一下我们的最优化问题：\n",
    "\n",
    "$$\\min_w \\quad \\frac{1}{2} ||w||^2 \\\\\n",
    "s.t. \\quad y_i(w^Tx_i+b) \\geq 1 ,i=0,1,2...n $$\n",
    "\n",
    "现在，我们的这个对优化问题属于哪一类？很显然，它属于第(c)类问题。\n",
    "\n",
    "因为，在学习求解最优化问题之前，我们还要学习两个东西：拉格朗日函数和KKT条件。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**拉格朗日函数**\n",
    "\n",
    "首先，我们先要从宏观的视野上了解一下拉格朗日对偶问题出现的原因和背景。\n",
    "\n",
    "我们知道我们要求解的是最小化问题，所以一个直观的想法是如果我能够构造一个函数，使得该函数在可行解区域内与原目标函数完全一致，而在可行解区域外的数值非常大，甚至是无穷大，那么这个没有约束条件的新目标函数的优化问题就与原来有约束条件的原始目标函数的优化问题是等价的问题。\n",
    "\n",
    "这就是使用拉格朗日方程的目的，它将约束条件放到目标函数中，从而将有约束优化问题转换为无约束优化问题。\n",
    "\n",
    "随后，人们又发现，使用拉格朗日获得的函数，使用求导的方法求解依然困难。\n",
    "\n",
    "进而，需要对问题再进行一次转换，即使用一个数学技巧：拉格朗日对偶。\n",
    "\n",
    "所以，显而易见的是，我们在拉格朗日优化我们的问题这个道路上，需要进行下面二个步骤：\n",
    "\n",
    " - 将有约束的原始目标函数转换为无约束的新构造的拉格朗日目标函数\n",
    "\n",
    " - 使用拉格朗日对偶性，将不易求解的优化问题转化为易求解的优化\n",
    "\n",
    "\n",
    "下面是拉格朗日对偶问题的公式，不想推导的记住就行了。\n",
    "\n",
    "想看推导的参考：https://www.zhihu.com/question/58584814"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----------\n",
    "\n",
    "\n",
    "我们考虑优化问题如下，记作问题（P）。\n",
    "\n",
    "$$z^* = \\min_x f(x)   \\\\\n",
    "{s.t. } g_i(x)\\leq 0 ,~\\forall~ i=1,\\ldots,m,x\\in X$$\n",
    "\n",
    "问题（P）的拉格朗日对偶问题（D）写作\n",
    "\n",
    "$$v^* = \\max_{α\\geq 0} \\min_{x\\in X} \\underbrace{f(x)+α^T g(x)}_{L(x,α)}$$\n",
    "\n",
    "其中的函数L(x,α)就是我们熟知的拉格朗日函数。\n",
    "\n",
    "----------\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面，先将有约束的原始目标函数转换为无约束的新构造的拉格朗日目标函数\n",
    "\n",
    "公式变形如下：\n",
    "\n",
    "$$ L(w,b,α) = \\frac{1}{2}||w||^2-\\sum_{i=1}^nα_i(y_i(w^Tx_i+b)-1)$$\n",
    "\n",
    "其中$α_i$是拉格朗日乘子，$α_i$大于等于0，是我们构造新目标函数时引入的系数变量。\n",
    "\n",
    "其中$αi$是拉格朗日乘子，$αi$大于等于0，是我们构造新目标函数时引入的系数变量(我们自己设置)。\n",
    "\n",
    "现在我们令：\n",
    "\n",
    "$$\\theta(w) = \\max_{α_i≥0} L(w,b,α) $$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当样本点不满足约束条件时，即在可行解区域外：\n",
    "\n",
    "$$y_i(w^Tx_i+b)<1$$\n",
    "\n",
    "此时，我们将$αi$设置为正无穷，此时$θ(w)$显然也是正无穷。\n",
    "\n",
    "当样本点满足约束条件时，即在可行解区域内：\n",
    "\n",
    "$$y_i(w^Tx_i+b)≥1$$\n",
    "\n",
    "此时，我们设在$y_i(w^Tx_i+b)>1$时的$αi$为0，在$y_i(w^Tx_i+b)=1$时(也就是支持向量点)的$αi$为大于0的数，那么显然$θ(w)$为原目标函数本身。我们将上述两种情况结合一下，就得到了新的目标函数：\n",
    "\n",
    "$$ θ(w) = \\begin{cases} \\frac{1}{2}||w||^2, & \\text {x在可行区域} \\\\ +\\infty& \\text{x在非可行区域} \\end{cases} $$\n",
    "\n",
    "此时，再看我们的初衷，就是为了建立一个在可行解区域内与原目标函数相同，在可行解区域外函数值趋近于无穷大的新函数，现在我们做到了。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在，我们的问题变成了求新目标函数的最小值，即：\n",
    "\n",
    "$$p^* = \\min_{w,b}θ(w) = \\min_{w,b} \\max_{αi≥0}L(w,b,α)$$\n",
    "\n",
    "这里用p*表示这个问题的最优值，且和最初的问题是等价的。\n",
    "\n",
    "接下来，我们进行第二步：将不易求解的优化问题转化为易求解的优化\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们看一下我们的新目标函数，先求最大值，再求最小值。\n",
    "\n",
    "这样的话，我们首先就要面对带有需要求解的参数w和b的方程，而$αi$又是不等式约束，这个求解过程不好做。\n",
    "\n",
    "所以，我们需要使用拉格朗日函数对偶性，将最小和最大的位置交换一下，这样就变成了：\n",
    "\n",
    "$$d^* = \\max_{α\\geq 0} \\min_{w,b} L(w,b,α)$$\n",
    "\n",
    "交换以后的新问题是原始问题的对偶问题，这个新问题的最优值用$d^*$来表示。而且$d^*≤p^*$。我们关心的是$d=p$的时候，这才是我们要的解。需要什么条件才能让$d=p$呢？\n",
    "\n",
    " - 首先必须满足这个优化问题是凸优化问题。\n",
    " - 其次，需要满足KKT条件。\n",
    "\n",
    "凸优化问题的定义是：求取最小值的目标函数为凸函数的一类优化问题。目标函数是凸函数我们已经知道，这个优化问题又是求最小值。所以我们的最优化问题就是凸优化问题。\n",
    "\n",
    "接下里，就是探讨是否满足KKT条件了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**KKT条件**\n",
    "\n",
    "我们已经使用拉格朗日函数对我们的目标函数进行了处理，生成了一个新的目标函数。通过一些条件，可以求出最优值的必要条件，这个条件就是接下来要说的KKT条件。\n",
    "\n",
    "一个最优化模型能够表示成下列标准形式：\n",
    "\n",
    "$$\\min f(x)   \\\\\n",
    "s.t. \\,\\,\\, h_j(x)=0,  \\,\\,\\,\\,\\,\\,   j=1,2,3...p, \\\\\n",
    "s.t.   \\,\\,\\,\\, g_k(x)\\leq 0 ,k=1,2,....,q,  \\\\ \n",
    "x \\in X \\subset R^n $$\n",
    "\n",
    "优化模型的拉格朗日函数为\n",
    "\n",
    "$$L(X,λ,μ)=f(X)+\\sum_{j=1}^p λ_jh_j(X)+\\sum_{k=1}^qμ_kg_k(X)$$\n",
    "\n",
    "其中$f(x)$是原目标函数，$h_j(x)$是第$j$个等式约束条件，$λ_j$是对应的约束系数，$g_k$是不等式约束，$u_k$是对应的约束系数。"
   ]
  },
  {
   "attachments": {
    "94a14d44-ffd0-4277-abb0-a53cae2c160c.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAADzCAIAAAC0QmJ5AAAgAElEQVR4nOydZ0BTVxvHTxL2NuwlyFAEFHGhIFjBUaAi7iqu2ha0WqWODrWttnaJWhyvgzrqVrQqKqB1W0XZIEsIeyQQIJtA5n0/XAwhhJCEgELO7xM56z73Hv73nnvuOc+DQRAEQCDvHurOjTsCV28I8BrRY1E2s2xB8FeHks47Gxn1g2WDjMxniZcfF2PftRkQCAAAcLnM0gq2g6t9jyU55Or/HvzXygO5aXnVzbx+sG2Q8TLlpc+kiRj4zIe8D5TlP//z/NPDv29/14aoCxrv2gCIulOVn0bTcyxOy5o+c/q7tkWNgMqHvGMKivJp+Po6ttnWQJ93bYsaAUf7EIg6Amf4IBB1BCofAlFHoPIhEHUEKh8CUUeg8iEQdQQqHwJRR6DyIRB1pH9W8jT8FfvX0zdUKwy5nGUQNid84YJZ+gAAAHi8mgd3nj1/+vjByzwAgNnQUfMXzJ36YYCzsWG/GAaBqCn9vZKH2VT87eZtNuNCt29YLUosTH+4akkUx8jl/O0zo2wt+9MeCEQ96e/RvqHZiLVfrE04de11bb0osbKiigKAt6+3O5Q9BNIvvIP3fJdRzhY6lFfP0t8m0FPTUgHATJrkg1O2zdTU1KKiItXYB4GoAe9A+Rpamsamhg0NDehPDof2Or1AS8vOe7yX0m0uXbr0559/VpGBEMjgp39m+Bof3b6fkHC7CWdrxuEFfBpCb2ZQKBQ0r6qwsoFEth4x0d7Jtl+MgUAgfa98bmvVz9t3vyxv2737t0mejnxezRcLo+ubmn3weLRAWlo6WSicM3aMjZZWXxsDgUBQ+na0jyCkvbv33Xpa88POHyZ5OgIANDQtHD1saQCYm5sDAIRCYnpqBgDmPj4T+9QSCAQiTt8qP/Np5o0rSdOCpwWMcUVTEITS2NCEwVhZWloAABhN1KKcEgMj49ETPPrUEggEIk6fKr/5zp07LE27ufPniJL4fH5dNcnMzsp78igAQHF+aS2D4eg+zNzctC8tgUAgnehD5fP5rYSiMmNLU8cRQ0WJlfkVxGqSj98ERwMDAEDqq1QuABMnTpSm+9aSOmbfmQeBqDN9qHweh0ttpJtZ4HV1dUVp9x88aEQsAoMCAQB8fm1WejYOZzPBZ1zX6hUFOc/+e9Z35kEg6kwfKl9TW8vcGq+hidPCYNCUyoLMa+duBoZNnz5tAgCgvoJUSagxGGLs7D5Moi6fV3PxxDWfKVLuCBAIpPf0ofI1NOxC5gQ31VPobDYAoKk678fvd7oEzd2++xs9AAAAGamZtW1tOvo6OIy4GcK8Vw+2b9qe2dxma2fVd+ZBIOpM337PX/TJohYme/uKr2ysW+maLvPW7QoO8tECgNlc8u+9e5dO/GNqaspnVS+ascTW3sBAByvgcRiNRDKTDwBYtmYrvk+Ng0DUmEHiddvZ2dnHx+fixYvv2hAIZGAAI230P0jqo1vxV64BJ2dOVrXDtMDlny+xwim9WQkCUQao/H5GcP74nycSMg8c3e/lYC0UEvf/tH/zF9kxh36Di5ch/Qn0xtWvFGf9d2z/X3Pnz/VysAYAYLE2ESuX1qc+PHH4zLs2DaJeQOX3J5zEO0kUnG1AkJ8oycrReoyPZ8K1228am9+hZRB1Ayq//2A0VTy48wRvY2Zq2fHVAoOxdnFxppOb8jIK36Ftg4LkKAwGg8FEJaus4CAGKr//oDXTWSyWiamx2KJGAACws7MDgJydnfOuDBscJEeFxAEQmYQcD5bMkFR58HGEEOsL4kLUWPtwhk95BIK6ixcfyVPS1NozZLo3g8ZkcrkWujr6nXP19PQAAFQqtQ9sVBuSo0LigG8sQVL2oPTA7jgp5V02nomNd40OiQrvcqdQD6Dye0VK4s2sitoei3353a8AACqFygRADxW6GPr6+gAAKpXKBQDO7ysFKu/IHRtdxBKTozAh0kTfjsvGHZHRIXG7D2wN7lRNTYDKVx4czvbo5X/kL483HWIMALuV3QaAjlg6u5UNADA3N4OyV5LSxPgU4Bu7VcGHd/DWWN+46OiY5I1q+NhXjfJfvXqlknbkwc7Ozs7Ort8Op0IMjQ31tLS4bdyWzspvaWkBAJibW7wrwwY67cIPlXhyBx9HkOMAlB7wc41OkVrRJXSRb3RK3M3k48FqJ33VKD8iIkIl7cjDhg0bNm7c2G+HUyEWdhbmFvjGZnorhwO0tUXpJBIJALyTk9M7tG0g043w5QGVfn5JKQhWtwG/apRfVlamknYGFgJB3fqIDXK+5y8LD9LRt/cLnHzycmYzmWJnb41mIQiJUEwwtbXzC4KeCJWDUJgCQORwpaTrMtwTgLhCAgBQ+RAF8A0N95WjGN4A/YCvMXPGzISLSc8fvfRaOQ/Naqol56TmTwlbOswQhhJ8B7i6y9OBgxCofOXB4WyXL1+uUJVRkwNWrl164Z9/Aqb5jRpqiSCkC2cv4ZzHbPhmXR8ZqSb4urv2orY6Dveh8vsZ7U83fufmduPsL1sFDq78glorn0mnL/1sran5rg0b2KT0asDuqdyrwoAGKr//wfrNmu83a/67NgMCQPskgee7tuIdAFfvQgY6ru6+6IBdCUpL8nv7qjBAgcqHDHRcQhf5gpT4REnpt+/LQT/mx4VI26RTmhifop6D/f4Y7Tf8FfvX0zdUKwy5nGUQNid84YJZ6ML1xvL8h08fxJ++XsfhAICbEjzXf9LU8A8nIgjp7j/37ibefVVYoaFtt3DFwjWffazTw1Eg6otL6CLf6Oj4xNKNCi7DRYUfGa52y3gAAADpRxiNb75YMW/3gZPiibkp9ya7unp4BhWSm8TTzx7dExgW8bqqXp6WnZyclixZokpbIQMJQqwvACAySbFaSZEAAN9YQt/Y9J7Tr6N9Q7MRa79Ym3Dq2uvaelGix0SP0eM92lrpmS+yRYmFaY8SH9bEHto3aqhlf1oIGZi4bNwRCRTcdSt1l4/60N/v+S6jnC10KK+epYtScDjb0NkhAFBv376D7lMllWfv/uHE+u83QNlD5CX4eJJC2i89sDI6BUQmqeFeHZT+Vr6GlqaxqWFDQ4N4on+Qr7Oxcd7L3OKicjaN8MeO/Yu2bJwyWj3vxRAleat9+XzyuKq37vtlhq/x0e37CQm3m3C2ZhxewKch9GYGhUIRL2FiMSwwNOCvi7fvJv6bTMx2C1kQPt2n7w2DDDLaN+epsOAgpm+Vz22t+nn77pflbbt3/zbJ05HPq/liYXR9U7MPXiJ8jl5YWNity4nn444ti1y7Ztkc6c1BIBAV0YejfQQh7d2979bTmh92/jDJ0xEAoKFp4ehhSwPA3NxcovBwb1cPbzcMYuDh4d53JkEgEJQ+VH7m08wbV5KmBU8LGNO+RApBKI0NTRiMlaWlpBeKtEfpVVghgtTfuZPY1nc2QSAQAEBfKr/5zp07LE27ufM7hu58Pr+ummRmZ+U9eZR40Yr8l0f+evzrru1ORkaZz9KLqoh9ZhUEAgGg75TP57cSisqMLU0dRwwVJVbmVxCrST5+ExwNDESJNFLBrz/ERe3YMG74qMCQgLbWqgdJcjm0lQNK1rOk+VOnfvzpV9nFDQCwctOTQ/3nnrvxgK9sFS6n6tOlX5F4PBVZCIG8G/pK+TwOl9pIN7PAi/mW591/8KARsQgMCuwoxq2J2XUw9Is1U0a7AGAwO2y2BRZ7L/nfOg5HFVbgxwaERHy2mE5n2A63AMCgJL/sq727l8+d3v3EZrdVEF7N1qjt+VRk1KjRvNKs6HW/ULptBAJ53+kr5Wtqa5lb4zU0cVoYDJpSWZB57dzNwLDp06dNQFOEQuLeXfscAj4SfcMbMXaEu9eI6jeV2al5qrJk2oypSFVlYc6b/5KvCsxHTffxUK6Kpqb9z79+Vn73jokFKbWGs+WPr/E9NgSBvK/0lfI1NOxC5gQ31VPobDYAoKk678fvd7oEzd2++xvU2zyCkA7/fjiLKFyydLZ4rfE+44RC4vVrN9iKHM7Hx2fkyJFSs4ZYDZsSNPLStYsZtYKPP5oqT2tSqwiFxAvnL5a1GGL57qSX96+ev6GIgRDI+wUGQZA+ahpBSKcP/v3ov0ob61a6pktI6KzgIB/UpfzzpGvXE66/yCkbYuexePnCST4TR1qb1pe8Tn+ddi/xbnphDQBg0vS5kyf5fDR7moHsw8hBQeqDqC3Hzif/jc4vFGY82vx9TNdi7hOC9v20RWoVAAAApBcPqT5TjXZ+c/H7mGWpj6gB03sePkAg7yd9qPz3BC6nKuanhNLCfz9YtmHl/Jm9rCIQ1F0+VxCxSq52IJD3lkGufKGQeGDHufnfb0iLP3nxQeHpc0eMe3rmS60CgQw23vEu4b6FfiZ2b241CUGQZmLezAkzXhWV90EVCGTgMWi9cbHqy47s2fu8iqEj1AeguaG+wVCPe+TQsZd5VSqsAoEMUAb5aB8CgUhl0D7zIRCIDKDyIRB1BCofAlFHoPIhEHUEKh8CUUeg8iEQdQQqHwJRR6DyIRB1BCofAlFHoPIhEHXkvVI+c++uLfMjviRyue/aEghkkPMeKb+xuvxRwn+zQmfaaGm9a1sgkEGO8spvqip4nP5GhabcS7rPMHWYGTZNhW1C1IPkKAwGg5Evop58BQc9Siq/Iv/l6sWR61au/e91iUrsaGOVJyUkB3/0obhDbjGQ1EcJm6OWb/5j5/rFq2OOna8XCFRyXMggIDkqJA6AyCTkbYDM0gN+GDHEdB58HCHE+ioYb3twoty2/vMxBwlU2tXThw6cjFeJn4D7N86NnRCSVVErLZN/7ljM1OCPcyqJCIIIBHUxP25eGrmpjsNRyaEhA5ukSACAbyyhc4IEkUliNQixvpJJ6ocyz/xWVrnOqHEuJsbjJ44jElUQD0cgqEtISPCbPmWMo23X3OKs/47t/2vu/LleDtYAACzWJmLl0vrUhycOn+n9oSEDnNIDu+MAiNyxsVPMdTFVo/eBuN0HSkW5Lht3REokqR/KKF/XwGn+LF8AgL6Rnq6OTu+NKEovyshsDgsPw0jJ5CTeSaLgbAOC/ERJVo7WY3w8E67dftPY3PujQwYwpYnxKcA3dmuwWFrwcdGwHwAQfDwpEgCQUkgQL7I11hekRMeo8ZC/V3P7fC5fpyOEjtLU37hxw2Gi1+gJUqLoMpoqHtx5grcxM7XsCGyBwVi7uDjTyU15GYW9PjpkAIMKf1GoS89FO+MSusgXgLib6iv97sNMyUHaywx9fTN5Sp4/FnP+5sPucpkVFWwHhxXB4jdu4DzK/38x22nNdBaLZTLMWLfzLcbOzg4AcnZ2zsJgf+WMhwx82p/4soWffDMOABAZ3um/C7iELvKNTskvKQXBCt82BgXKK7+y4OWhmGNLt/0qZ3kmkyk1XUgmNxpaWLa0SGTzBFwAAIPGZHK5Fro6+p1z9fT0AABUKlVRsyGDCEJhCgCRw2VJF50HkHgfAAAAl+GeAMQVEgCAylcAPr825o+9DRYm+vp68pRftmbrsjVbu6Yzm98sn/fZ/CVrt6yJkFqRSqEyAdBDhS6Gvr4+AIBKpXIBgOt+IN1QemBldArwjT2zsau8Xd1934FF7w3KvedzTh86/TiD8tXKj3T1e/We//T+82rOkA/DZnRXAG86xBgAdiu7rXM6u5UNADA3N4OyV3N83V27ySk94OcanQIik15I0X07+SXqOr+vjPIznt4/E3cxYs1yK80hurrKz+3z+bW3b93+YGaAp41Fd2UMjQ31tLS4bdyWzuktLS0AAHPzbitC1ITOk/YdJEehshef5++Kp8xXhcGMwspnNBbt/W2/0SjflVERdBpdR0955ee9zMvOY4XODpFRxsLOwtwCT22mt3I44ukkEgkAvJOTk9JHhwxeSg/4YULiepA9oTClH01671DsPR9BSIf2H3ldjRw8/ZWdtjab1aqtow0AQBh0jFFH+LmEC8fvZZOP7P1elCJ1bl/Ir6NxjGO+3ywlxt3buX0dfXu/wMknL2c2kyl29tYiMwjFBFNbO7+giQrZDxlcuLr7ogN28fl50SBf9tO+tCRf1qvCoEexZ37y1eQrV5OXfB4x3ccTANDSwmIxWQJB3V/XnokXCwwMXPnJSom6TEmqyysF5uZIl/R20Ll9ADRmzphpJmx4/uilqKmmWnJOav6UDyYPMzQEAAi4Nc9u3S+soilzASADGJfQRb4gJT6x07t6+7M8LgTTGb9OS/ZKE+NT1Hmwr8i6/ZritJljvWfNX1Xd2oqm7P/56zXf/hh3cOeDtHwFVw1zD/+xY3rYclFTMmk7Ebt72uylr6vqEQQRCol/7v5m1vxVlS0taPbhP3Y4OzuP9Z2dXlqloBmQAQ4h1rfzon3py/YBkCgFl+7L+8zn82v/jDnA1B66eesm+7crdqcGTK3LILTqegRN8EBTeLya8yf3L5g250WB9HkXFFpDyb1bD2fP+chersW/2p9u/O7ndfPO/rJ1068/b1y2k2044vSlOIe3X/osza2cLU01GisLclS5axgyAECf+p2W4QYfl/6vLj7DnxwT3WXRr7qh8ntJC6N0yezVsh/m184cnjBlTlFDkwqPm3Ah7mVhmQobhAwM0Ge8Io9v+MBH+iKKdg2hRtvBQsbDnMeruZOQGPRhoJuFqaoO2kIlFNZpeY+EU/3qB7olR/4d9+jqnsgkmfN/gx/VKz8397W391gZBTKeZOQWC0LnqOzCk0qzD599tWrTMm1VtQgZULzVvnw+eVyh7gEAAIMgiEobpO7cuCNw9YYArxFSs4VC4jfrvmnE2R04/Jux1BIQCKTvUfEzn8tlllawHVztuytQnleW/qIqPDwcyh4CeYeoWPk1JdUm7i4OkvtrRDATbt7Sdxs+McBbtceFQCAKobLRflV+Gk3PsfjJZS1Hn/BAn25KcRmMNgAMjIzeI2/fEIga0ivPHOIUFOXT8PV1bLOt3coeAKBlZAQ310Eg7x6Vz/BBIJABABx1QyDqCFQ+BKKOQOVDIOoIVD4Eoo5A5UMg6ghUPgSijkDlQyDqCFQ+BKKOQOVDIOoIVD4Eoo70k/IZTUW79sW9qYPucSGQ94J+WrfPaCpaOGtZDTJk3qJ5S5Ys9nBQ1A8XkvroVvyVa8DJmZNV7TAtcPnnS6xwuD6xFQJRB/rL4R8v+7/kLWuXjXR2dhs17Zuf/8wm1Mtdl3/uWMzU4I9zKokIgggEdTE/bl4auamOw+k7cyEDjXZn2z371ZS74OCm35SPws95fvfrL1aMdHYePjJg0w970gpreqzzJvOx3/DhsSeuiFKI5ZmBXl679sX1pamQgUQXB7yof10RnV3yQ+e7/a58FEHOi7vfrFvp7uzs4uL35Xe/pLwu775w275dW0eOnJZVUStKEgqJmyIjxk6eXURWpd9uyEAlKVJC3FKjbYgrHWr/ncztY718Z/1++PTlc0fnf+jyKP7UinnL127d9SxLSnAORlPFgztP8DZmppZ4USIGY+3i4kwnN+VlFPaj2ZD3k9IDu+MAiNzROVZ21+d/3M0O17wuG3dEAhC3+4C6BtFWcG7/0c3L/4v5IfrzJZt2dETBLEh7OG3atAsJjxQ8NGbU5Bm/H/r78vljC4KHP79xbvWiZZ9Ff/8wtUC8EK2ZzmKxTEyNdXV1xdPt7OwAIGdn5yh4UMigozQxvku8nODj4uE0XTaeifUFaOzNjiJbYyWj86gViik/MPzjaR9MTXlc4OXt9TaNd/fu3YYGjOfYkcpZ4Dlp+m8HT12+cGxRiEfanUtREStWrf/27vNcNJdBYzK5XB1dHf3OtfT09AAAVCpVuYNCBg2o8BeFygyNSShMAZKFXEIX+XYeCKgVCo/2i4resLTMPL3bdc5sLn+c/NTB3dHSxqI3dnj4TP/6668joz4GgP787rU9e/acjX8AAKBSqEwA9PT09AAQCoWi8vr6+gAAKpXK7c1RIapGvI/6BXmEnxwVIuV1oF36nQYCaoSiym/KysyydbW3sbdCfz+6++RNI8V7rHdvvq5Ta4oO//H9nJB5uSW1I8d/8P0fRy7furBi0XQAAN50iDEA7FZ2GwC1tZUCgQCtwm5lAwDMzc2gQ8/3CvE+6hcIhTKCYSdHYTAYDCYEJCEI0jWqjstwTwBSCmXFfh28KOZ7l9PGys8sGhU411pTEwDAbH6TRarEYKwmTBjfqRzCrCsFtq6GPTZIqS68cPHitQvX61q5E6d9FDw3JChkpngQDkNjQz0tLW4bl8Zp09DQxOFwAACBgN/K4QAAzM27HWgIBHwKpUkg4FtZ2Sl0ju8JbHYLDofT1pYn1vD7Akesj3qDUCggkWptbR1UYhUAIC4EEwd8YwkvJJ76ru6+3dQY/Cim/KqiquaGtkXjxgEAAOAnX3yiR2/TNzbwHOcmKlOW/urKzYv4kYFrXMNlNNVcVXDh4sVr528QOcgY/xkbPv7Y2916qLWdhEEWdhbmFvjGZnojsWakvSMAgMVi1tfXsLk8APBOTtJDaDY21iOIsLm50choiEIn+D4gEAgaG0kNDcRhw4YPLOWTySRra+n32YqKktZWtpvbaCy2h2Emm91SX1/L4ynwGufr7io9I/g4ghwHAJQe8HONToleeSBUUvsAHe4Hy5wlGJR0dAOLxZBRDs3Nzs6haJqgL/m5Lx4KPJyz0nId3YaZ4jsE5jzBlcsQek/06q6p5qqCg79uCw9ZePDkHfspwX/+de7K6YMffTBWG4vV0NCUKKyjb+8XOJnHaGlpaUVzDQwM8XgcpZliamvnFzRR6iHMza0sLGwGouwBADgczsrK7r3SvOz/DRQOpw0rrQdRNDW1NDU1MRhMj+3o6embmVkqZF6PA3b0G1435bp7VXgHyHOdVdVyu/KFQmFzM7m7am9z67Ozcoa6OVjYmJMrc+/nNE+xM6qvqffx8THi1x47egMtzOGwqsgYe0fbru00VeYf/OW78OAFB0/ddwmc878z5y4c3zd72ngcAI2N9RYW1tIOrjFzxszJnuY52R0TMfRGWkNd45QPJg8zNAQACLg1z27dL6waVNuB5BFJ/yD7f0NE9z0IAAB2do6urh7vz0mhEApT3rUJHch5nVXVMhYAwOPxamrKu5uVFeXyeLyyNxUmeJPa53cPn3y+bM3imqo6Ig/v6uF06o/roZ+0T6BU5JfrOlgYMspPHPp5f9yl0kYWAIBNLz2Aav70M/fgxScunzt96LdZfqPRKlwuBwCgqSl9tm7EuEmTAibEX/0nr7oBAIAgpAf/PsQMsdzwzTq0wLHYuNWb1i5fuiKjrLqX1wgigez/DRGyexBBEC6X0zcPNFd3KfPzyVGdAmqXHvALiQMARIZ3nuQrLcmX8arQr8h5nVXYsgaN1szlcoVChMNpI5NJAAANDU083gzNFs9tbub6BflnVdCSs0jrf/pCl8XQ0DdY//nMx0kPVqyYj5Dryng8PN4sLS198uTJLBrbyWd+4ER3Pp9XUJCNA+z7N154h69YsmSJ7+hhEkbIflw0NlLmfrzKztLi7C9bBQ6u/IJa12n+s2d5W+GwdXVVAoHAwWVEkN+EnJcFBTlvxjsP7fFacLmchgaipqYWBgM4HI6lpY340Bq9DtraOnw+TygUaGhoNjWRHR1dDAyMemxZKBTU19dhsVgsFtvW1mpqaqGv3zHNyeVympvJOJyGQMCnUpu4XO7o0RNE7708HreurhqDAVhsp4m91lZ2bW2lUChwdHRtaCCi7ZiaWgwZ0vN+R3nqotNpQqEQg8Hw+XwrK1sdnfZFU7L/N8SR0YMMBq2trZVOp2AwWBeXjmsouxcAAG1trfX1dQgi5HK5ZmaWpqbm0s4I67v+m1G/vUws3bjRpSNX1/Ra3FziXxhgYmKckZGTlwcA8D24ZWZNTXlzc6Ox8ZChQ51xFYmFWv67Vjlp11RwLW20tLTl6UGpNstznWX0fo/XuampgcfjoVOnbW2t8h9XVsvo+sbGxvry8uLulvh2l0unU/PyMojE9l03fD6/sDDl963fXLsVd+9lDprI4bTl5KTl5TztbnMOl8upru523b7UXAqlsagot7a2UigUIgjC43Hz8jJvXzn1srBMvFhtbVVtbZVEXR6PW1iY09bWKjKvsDCHy+WiPwUCQWFhDpfbvguwrq6KSKxmMOg8Hrc7C0UIhcLi4nwKpRH9yWazcnPT+Xy+qEBJSb7op0DAf/MmT2QGn88vKMhmMGhv67bk5qbT6VT0J3qda2rKBQIBegqvX6fz+bweTeqxrlAoLCkpoFCaRMfNy8sQnT6K7P8NpKceRGlubiQQCkU/ZfcCanNVVRl6ubhcTm5umqgLJM+o+OD+bZs/OFjcKTfzpL+mBgBAX19v27bNuh8cFK3pLy7Ob24mIwhCiPXV0tLcf7fj1GT3oDw2y+gjGb2P0t11JpFqS0uLRD8ZDFpxcb7opzz/G1Jb7u26fSwWJ5rOxeFwetqaujZDtbiCf65eRxO1tLQ9Pcd6jPafMFL6rC+ZTJLxwO8ul8vlWFnZoe+NGhqaulr8plZd75HSp/rFIZFqjYxMRI8XLS1tI6MhJFIN+rOlhYnD4USjVkNDYzqdamho1N3ElTjNzWQEEQ4Z0n6rxmCwmpqaQmH7x20EQVpb2aIXXSwWZ2fnKKrb1NSgoaFpaNj+QVNXV09bW1u8cSwWZ2vriD4iNDQ0NTW12tpaezSpx7o0WrNQKBQ9InR19fT0DJqaGuRsGUV2D6JIvODL7gXUZnv7YehTTlNTS1NTq62tTfoZDQ+21WeV7z2eLJ7r/ckzLg9BEBarZfnyFY13Volm8fB4MzqdCkByTHSK67yDMyZ27AeR3YPy2NzddZbd+7LR0tI2NOwYK2lr67S1scULKPe/0VvlS3y8baipN7U2+TAslFNUnFFak56cTAcAh8N1N7WDjqi7m8eWkaunZyA6NKk0+/5/ZaMmuGl3LdcFGqTbLMsAACAASURBVK1ZYtxuYGBAp1NEP4XCTq5KsFh5v07T6VQDg461CDo6uiNHeoluIhgMxtgYX1KS39TUIBDwAQD6+gaiU2My6QYGspY/SFxDDAYj/4IZGXUZDJqeXqeF0draOi0tLDlbBj31YHf02AsSNmOxWJECu+S6eI0bqa2dEPL21V72tRoyxJTJZBAO/h4HwMpVM0eKfQCS3YOK2ix+XNm9Lxs83szCwobH41IojfX1tY2N9UhnbzrK/W+oLIo2Sk5Wjr3HOBzOdvX6eT9t3735203ohWQ0FX25cktDF4M8PEZUV9cxmVL+1UIXfzZ/1kRz8x4eJgAAaxfvRXh78btvdwiFQoFAIDERpampJRAIhEIhFos1NDTW1NRkMuno45dCaezxaSaCz+dpasoaGjg4ONPplMbGhrq6KmPjIdbW9qK+Fwj48gwrVA6fzwOA39hYL0pBEET0ni8PZDJJnj4Sp8deUKg1AABwDDw471DwHyFR4UiMXw9lcTgNQ27+jlMUbFTS4pGI+KNLRg/23mYZvS8bPp9XXV2uoaFpbm6Jx5tzuRxFB2VSUbHyl3z2ZXl5CRaLDQhZFBCySDyrtbWVyeGIp2hrawGAEIkkqU1hsYDP58v5XygQ8OVZOobFYrFYnPjTAwDA5/PRGR0AAIIg2to6bDaLzW4BAODx5qIReI+gkzeyyxgb442N8RxOW1MTuaSkwNXVHT1BHE7Sqv4Bh9PQ1NQyN7dSrjqfz5e/j0T02AtK4LvtYOQfoXEhmCHJ1E8kZ5A7kxwVsunJuHGfJcb4GvB44jkyelAlNnfX+7KpqCjR0zO0te156loh2pWPw+HEhxAsFkN8YCMjV8KLH5vN6jpqFQgEhqZu8Y8kt/GSSDUmJqa6unpSLUNzu7Nb4itFW1ur+BysDIyNTdjsFvGzY7NZxsZDRH+zWAwHB2dtbd2uPYrO+ujpGUh9eTEwMGQyJT9cNTU1oOtSGAwaBoNB7yPa2jq2tkMxGECnU9G+19MzlHg34/N7uInIaZVsDAyMaLRmiUQ2u0X8FUBG7zc2kpS7a8juBaXwRdfrMRg0kvRHyVuCjxd+iBQW5hhSm4xtO71sy+7B3tgsu/dRpF5nHo/X0sKysenVQmapLbf/c+vo6KEzjQAAGo2Cw3UaC8jIbWtjiy4WejOTWCfP5XLy87MIBEkXGgKBgMvldid72bkAgNbWFg6nfcqHx+MymTQ5h+XW1vYUSpNocSifz6fRKNbW9uhPPT0DLS2dkpLC16/TCwqyy8qKGhqIorsMkVhNIBR291phZmbF5bbR6e0bhxEEIZFqRP8oWCy2vr5OvAOEQqGengH6t7m5JYvFEM3cNDU1oNOz8pyRbKtkg8eb8/l8KrVJlEImkyRued31fo99JAPZvdDXYDAYY+MhfL7kOFF2D/bGZtm9jyL1OmtoaGhoaLS2toiKieRGoTTKeb5SW8bt3LkTAKCpqSkQ8EmkWhaLoaWlZWRkIl6zu1wOp62trVVLS5tGo9BoFCq12dbWQeKZLxQiVGqjjo6uxPdnMpmIx5t3t/ZDdm5bG9vYGN/SwmQw6HQ6hU6n2dk56ui0/wuinzFaWpg0WjOH04YgwpaWjkciDqdhaGhcX1/HZrew2Swmk2Zr6yB646LRmhFE6OIy0sLCysDAWENDi0ajtLa239rR5ShDhphJzIqhYLFYExN8Y2M9ldrEYjEYDKqJSUdJoVBIItVQqU1cLqetrZXJZGhr65qYtE8s43A4Q0NjIrGGwaAymQxtbW0+n8/jcbW1tQUCQUMDkc1mYbFYdFxDJpNoNAqfz9PT09fQ0JRhVVtbq+y6GAzGxMSUQiE3N5OZTAaDQTM2NpFop7vel91HEqB3atG0uYxekG0zn89XOld8JgVBhNraOrq6nc5Udg8qbbOGhqbs3pdxnTEYjIGBcWNjPZvdwmIxmUy6vr4hh9PW0sI0NDQRCnv435DVspwPFqkwGDQSqWbEiFGKVhQKBTU1FQ4O0hdMy87tU4qKcp2cRohPvTCZjPr6GldXj/435n1Gnj5CEET0AtLQQOTzeSrcftdLiMRqS0vb3m8rHLj09quecjcOBAEyNs/Kzu1TdHX1GYxO6/9bW1uMjfHdlVdbeuwjGo2Sn5+FvigJhQI6naLoPpy+gMvl0OlUdBpPnWUPeqN8Nrulqamhra2VRKpV1BmD7J3n73Bf+tChTgKBgEisbmggNjQQ0Zdn+T/sqQ899pG+vqGurl5FRXFtbSWRWOPg4NLHfdruhSNKpnMtBoNWlXpl+fKlzs6/q6kXLhFdVwtCIAOOLv72xWh3vdvFGa86+9x+R163IRCVkhwVEgd8YwldHW61O+WQSESd8caFyB4gDG6g8iEDHen+9gEAACRHuUan+EZGdnG6Bf3tQ+VDBjjS/O2jGQf8QuJAZNKLre5SqkF/+xDIQKYbt9voKD8ySYrL3Xagv/0BA6W2YM8vp8gIAgBoLM+/fDp2XkCAj4+Pj4/v5p0xN++mAQAQhJR87e+Nn3zs4+PjFzA/9sRldKEfp61qzYIFF289lmizqiD1j/+dVZGzGCT1UcLmqOWb/9i5fvHqmGPn6/vV/7R6Il346Chf6ot/B+rtb3/AzO2/enBz07YYkpijCwRBclPuTXZ19fAMKuwcWvPs0T2BYRGvqzoCdXO51cuDP7yS+BRBEAShC8QK15fn7Nh2qKqF3TsDYajvd0JSJJCcpe8yc9/tVL6UymrDwHjmF2c9PR6fvmVXtEQ8D4+JHqPHe7S10jNfZIsSC9MeJT6siT20b9RQy6KMxwG+C24+SNXU1HUfMWr0BA9G45s1y1Z8+c1uUXnLYV4rFoze+9OBJqA8xVn/Hdv/19z5c70crAEAWKxNxMql9akPTxw+04tWIcqA+tWMC8G8BZ3cjwvBYPw6z+ips7/9AaB8Orlo/4//W7xqobWG5J5iHM42dHYIANTbt++g2yxI5dm7fzix/vsNo4ZaAgAcnOwXh7knHDoUn1zkNMwJ05y36+vfMJbjZgTOEm/H1TtglCXv4qmrytrISbyTRMHZBgR17A63crQe4+OZcO32m0bJzXAQ1dI7J5pqOtwfAMo/f/piq/2w6ZOlO/D3D/J1NjbOe5lbXFTOphH+2LF/0ZaNU0a3v/bp4V3WffvT6YTzPkO1quoq09/wfzh95ui+78NnTZBoZ8782UknzuVWy97hKR0Y6vvdIuFHP/i4xMC2Y7QvJdDG++Rvvz9RWPlCIfHbzz5bHbVdtEWwqe51REjID3uOylGb//L+jc1RX+46cuzHzVF//nW5oSc3wywa4d7tB+PHj+9ujbWJxbDA0AAut+Zu4r9//LTXLWRB+HQfsXxq+pM7R2N+PJWUPdJtpJM+fft3v5yKTyomSTrnx9uYWVny7t6+L8dZSAJDfQ9Q3it/+/2Mwj55GE3UrKysyUsizd+mFOUVpxEYi7aM7alq48Hf/kzKJB04sn+ExZBmUt6i4OXxty/6j522Z+dX3dV58TClpElrs7dn983qhYWF3bqceD7u2LLItWuWzRHPe5Z0f/WGXRFRX27fuubS3r2eC+evNTL5au2Wu3cDTp06IL49WkPDws1zxL2kexGfLbbWaLp4UdKPiFRMrT1Dpnujob4tYKjvd4DI374SAbLeI3/7/Y/Cyq8orqpnGowd7/02QZCammowxMhz3EjZFR/e+vfEyWff7v95hMUQAICuga6mhsZ4j0kyZA8AqKiowGJxRnhZvu6He7t6eLuRsygeHpJLNvyD/S+fuerp544T1L5KTTX3CQzxDzp39e9mraEGks1oWVpa1lfkNtSRrR2wKYk3sypqZZ8RAODL734FnUN9iyMe6hvG/O0bXEIX+UZHx6P+9hWjNDE+BYBI9RzsK678jMxMrLGe+5gR6M8Wevl/D1KGuXmbGcsORMHLzMhE9HXH+41BfzfXU9hstpGR7FoMIpGIwWJ09WRt80p7lF6FFSJI/Z07ibOD/cWLYjDW4/2sAQBN9ZRSQimhpBT4j7V09pK6X1Qfde1AbMQ5jT16+R+ZhnVCPNS3+NFhqO9+QGnptws/XNYn/0GMYu/5QiExNzPXYYQjfki7Y5Ynd58VkMg+Pj49+qkUCAXG5kN02jUsuPlPAsPAYeas6TIPx2ogkbX1dLR0utVORf7LI389/nXXdicjo8xn6UVVRKnFzGxHX7t8Y+2ni6Tmoujr6wuFjfVEhR2bikJ9t3ROb2lpATJDfUNUgEvooh6W4bpsfIEgkov5kmOipS/6VRMUUz6jiVqcXzZ27FjUsRajsaiATjHQtB87fkyncgizjsDsXFVz2bKI4dog40U24JCS4s/ceUXc+cuuqWPdZFTEYnXMzE057DYuR3pMZRqp4Ncf4qJ2bBg3fFRgSEBba9WDpG7fz43dHGU/e7lcLgB4ExN5ne2KQEN9U5vprZ2dC5NIJBmhviEqon33jUI772Ts8lETFBvtV7ypqmcZeY9FP7A1/3PqkRa/WdtiiKtHxz93WfqrKzcv4kcGrnENF69r7+YYHjG9orbm+vVGvIvfuRufWHT2FSutIt7axhoRVrS2tIEu0dx43JqYXQdDv1iDfsObHTb7VnzSveR/l65eaKstT9ANSVpaWBoaWhY25gJB3fqIDXK+5y8LD0JDfZ+8nNlMptjZt7vxQBASoZggI9Q3RGUEH0+KjAuJC4kK736ZvjilB1ZGp4DIJLkKD1IUU356ZoZAE2fvZAcASL6WYOzv+e9P10d4jLczYJ+MufjR5s8ssVjnCa7cs0LviZKf38/+7++4G+lHTx8eZS/dK5PUig4ODkLhfwwKAzh0CsstFBL37trnEPCR6BveiLEj3L1GPMutzE7Nsw0Yr9B5obBaWrR0tC2szQBg+4aGy7O8C2+AfsDXmDljZsLFpOePXnqtnIdmNdWSc1Lzp4QtRUN9Q/qUt9rHABl7dFCSozAhcUDNda+Q8oVCYk7may63/u+4c8/06BjrsR9PNDpMapo93/vGyfhxi+ZbYrEAAA6HVUXG2DvaSlTncXn1ldnzZi30n+4fGhr6wSxfCWf6Uiv6B/k67D2W/7poqnfHtwMEIR3+/XAWUXjql9kdZ6JhN95n3JPs89ev3QgMGK+oL2gEIRUVvJkcNMnFdAgWDFm+fLlC1UdNDli5dumFf/4JmOY3aqglgpAunL2Ecx4jCvUN6WOCUX/7Kiw4uGn3ui0PtIbq0/svhn0+6+XLkjH+wetWzkOErFcpLwsrqYFzl0x2b49sUpqbnVXbHDre5tzJo09yieZDh+H1tSi1hc+fVny5Y4XzEM2C1AcXLlzIyCKMmeJrKjZpL7Wijr4Bsa6soIoS9mEgWux50rUD+w4kPs3EaptgtbV0jPHmhnr1Ja8fP7798kUKtYXXWFdeXNlMZ/McRwyTf1KdRSX9tf/Ksg3rRzjYyF1JHI2xk6a44DFX4g7dzctPOn5L22nMzt+32+m8G4eCEEgPyL+5J/3J7TFes/JJZNnFzhz54+z1+0RC5sPUAjSl/PXL9V/8LNq1JhQSr5874jdixC8HTsmuiFJVlDprUsiL/FL5TVWCG+eOrlr/Xb1A0HNRCGTgo8DcfkZGho2rnZlFtxGvADpmTivT06jIa8IFTkTX1XAuXrzgNtHdRksUVdZ67rJPQ+bPZDKZMiu2M9Rt4udrQ86cOtspdLBKoZAKEpOLt+382lLZuG4QyMBC3n90gaA241Wml7eXbG0wKfS6NgMtruCfq9ffpmnr6euVlJSIFxMKm6hE5rixY2VW7GD+imW+TnpHjl2Q01qFaGOWH/rp9LJNa1xNTXouDYEMCuRVfs2bmtI31Em+PrKLEQrKLN1sPwwL5RQVZ5TWpCcn0wFYtGghLSP9xOVEIo0LAKPmTcaBXYdcAuctCJ4iu6IYxivXfemO5xw8Fa/gCfZAK6PswB93lv3y49RxI1TbMgTyPiNndC3evwlJrQD4fPSRlczIJId/32Hl/eGCWVOeJcXvPfNw87ebpnqjiqrPfpxT3djUqmPl4ODg6uVqJm/FTrSWN+g6qTJUi4BXy9a0g5/dIOpGr+LqQSCQAQqc0IJA1BGofAhEHYHKh0DUEah8CEQdgcqHQNQRqHwIRB2ByodA1BGofAhEHYHKh0DUEah8CEQdUdjrNgB8Wm0pyrApc3zchqneKAgE0scorPzytKwSYv6eH2KFNp7nl69V4pCU2oITZ1JXbfsEU1Hw8OmD+NPX6zgcAHBTguf6T5oa/uFEBCHd/efe3cS7rworNLTtFq5YuOazj1HXNpy2qo3LNgesWLc0bJqowaqC1MtPiteuWyHbd798IKmPbsVfuQacnDlZ1Q7TApd/vkT2JqW+h7bv+99eVLXs+99vyrn0QxBS8tXke6/Ie/Z/26NnUgG39vShM6mZqYQanoWr64wZM5ct+Ui3p1qQgYcS3jzYrLLQSRPXf7tbibqvHtzctC2GxOeLUnJT7k12dfXwDCokN4mXPHt0T2BYxOuqevFELrd6efCHVxKfIgiCIHSRA5368pwd2w5VtbCVMEkM/rljMVODP86pJCIIIhDUxfy4eWnkJpE3oXcCn1+7NmLeyJEfPM8nKFpXKCQmXjm5KCTA2dl54Scb2noq38Yqj1wUPnvJ55u2R3++NMzb2dnZ2fmr7/e8y/OH9A3KKP9N5mM/5zHnbjxQvOKTTyK3E3k88UQ+vzZqSbiz8/hzN+6LEgtSHy5e8KVI9oXpj/wnz79x/xWCNP4WvbWI3EQnF0VFzPni659FVUqynn75zW+NSpxPh3mP/YYPjz1xRZRCLM8M9PLatS+uF62qAC6jKqOoTomKNemFdASpKEjxc3OTQ/ltR/f8ePTcDdHvnJS7gV5ew4f7J/6XpcTR+52kSAAAGjRXRQUHMcrM8GVmZlG1h4we76FQLTq5aP+P/1u8aqG1RqdXDBzONnR2CADU27fvoKEnSeXZu384sf77DaOGtm/Fd3CyXxzmnnDoUHxykdMwJ0xz3q6vf8NYjpsROEvUjqt3wChL3sVTV5U4IwAAAJzEO0kUnG1AkJ8oycrReoyPZ8K1228am5VtVgVoGg4d56aMX1C78SONANAz1NXS1OyxMIdTX99m9emyjigJXpNnRaxeKBAQSwdCiPnkKNSZ9luv28lRGClEJQMAgo8jhFhfBYNzDDIUVj6CkDLTMx3dnSxt3gaNojFl1mjn/OmLrfbDpk+W9MMPAPAP8nU2Ns57mVtcVM6mEf7YsX/Rlo1o/AwUPbzLum9/Op1w3meoVlVdZfob/g+nzxzd9334rAni7cyZPzvpxLncapKiJwUAYDRVPLjzBG9jZmqJFyViMNYuLs50clNeRqESbaoMAb22oq2vD1JPIE+fGyRxhxg+fAQAQFOOG8c7JjkqJA74xhLkdaLvsvGMemtfYeW30FgFWcXjJoyzxGKbqvK2Ri0fO37cwk82VLe2yqjFohHu3X4wfvx4qXNlJhbDAkMDuNyau4n//vHTXreQBaL4GW+hpj+5czTmx1NJ2SPdRjrp07d/98up+KRiEk28EN7GzMqSd/f2fUVPCgBAa6azWCwTU2Nd3U7zWXZ2dgCQs7NzlGizlxSkPdz65WdL50yfPGbq45znfX04B88JUzwlI0qzWCws1tphmENfHpn/z6ljv2zfFLViwYrI7+g84qlDvy4NCwwKX/7LvlPyuV3tNliWb6zE5EjHnaE9KNfuAwNgPNMHKBFFu5LM1B0/cVwdIfPo1dTIn484Ou85cuYFi8ECut3OAb94mFLSpLXZ27ObfL2wsLBblxPPxx1bFrl2zbI5EtnPku6v3rArIurL7VvXXNq713Ph/LVGJl+t3XL3bsCpUwdE8bA1NCzcPEfcS7oX8dliO21tgaDu4sVuw+yJY2rtOdSQyeRyLXR19CUs09MDAFCpVHnaUS0eE4NiJgbF7v42h5DmMcZNodMJme7dczk5qKqqGu03ZpyvlJGa6tCYv3oNqSJradjKmWtDTv5+K+SztavXf/7n7r3/O3rYfpj9inkzemigNDFemeiYwVtjfeOio2OSN6phuB2FlZ+elobFmwy1x5y/U/jNti8MENKx0nJTO0sjE1nf1CoqKrBYnBG+2zLDvV09vN3IWRQPD/euuf7B/pfPXPX0c8cJal+lppr7BIb4B527+nez1lCDTgW1LC0t6ytyG+rIdk72AICUxJtyhscz5FGZAOihQhdDX18fAEClUrkA9H8wbKGQWFZWbuNiZ21nCQBZ/tNRydEppIKkG2lRv+7uB2fkFYTKer6pTjN36fZI9HC+vr5HzvxTUFAAelJ+u/BDFY6O6RK6yDc6Je5m8vFgtZO+YsoXConZmTkuHi7Pz6cs/26tIQAsGrMgq3jk1FA7WUEsGUQiEYPF6Op1G3Am7VF6FVaIIPV37iTODvaXKIfBWI/3swYANNVTSgmlhJJS4D/W0tmrqy9OfT19Pp/fQGwETvY4nO3Ry//IeWoFaQ+MAWC3stsAED86u5UNADA3N+t/2QMAWmgsQl65x6wF1pqaAChwOqqg9Xr8jdnrvwgJUM3wQTapaakYDGbC3CDRXYbH4yFv77wykSX8lGhXTDT6p28s4YXk2wAq/fySUhCsbkF1FY6i/eZ16YjRQyYsnoNGzih/U9lI4/v5TZFRSyhkNZDI2no6WjrS5VOR//LIX49/3bXdycgo81l6URWxu6bMbEdfu3xj7aeLuiugr68vFDbWExvkPqd2DI0N9bS0uG3cls7pLS0tAABzcwuptfqaakI1hcafMEGZAKG95N8bVxswNp99HNoPx+Lza3PScod5OLuM7AjKTCQRATAYNqzHRaKEwhQAPIf3JN2UaFdMl/k8l+GeAKQUEpQweoCjmPIJheUkJpPVrOfj5oimZKSnc/XMx04aLV4s4cLxL7b83HEMrI6ZuSmH3cblcLu2SSMV/PpDXNSODeOGjwoMCWhrrXqQJOtt1tjNUcbjl8vlAoA3MTFW5LQAAMDCzsLcAk9tprdyOOLpJBIJALyTk1N3FfuU7OxslhbeY8zInouqlLxX91+UsjdtWN0/h6uvIFaV1XwQ+IFouaRAUPf00TOLoS7+MyYp2WjwcfGZPfQTftf5PFd3eWImD0IUG+1npGcItYZ+vGwh+lMoJGZlZDuPdrGy6TTuDgwMtBojPvjHW9tYI8KK1pY20NnPPo9bE7PrYOgXa9BveLPDZt+KT7qX/O/S1QttZb0+dEtLC0tDQ8vCxhwAIBDUrY/YIOeL8bLwqX6Bk09ezmwmU+zsrdF0BCERigmmtnZ+QROVMKaXIAgpOzvH1tXeytYCKHw6QUoftyTr2eVHJV9vWyc+YUslIUOsMUq3KZu8nIIGoblvQMcHndcvcnNTKxZ/842jgYGMiiJ83SW/SkjQHmY7JT6xdGOXLwDqONxXQPkCQV1OZo6Ll4uDy1A0hdFELc4rm/H5HLyQeHRP4rJvP0eXlRtau/pYd6rr4OAgFP7HoDCAQ0eQbKGQuHfXPoeAj0Tf8EaMHeHuNeJZbmV2ap5tgDJDXFZLi5aOtoV1+w3GNzRcnls63gAPgMbMGTMTLiY9f/TSa+U8NL2plpyTmj8lbKlyC+Z7CZvZUpRNGPXhQuu3n9MVOR25EHBrXtx9Y+Y1wd2hPbJYZcGr04mvv/p+vfioqanm9eUHhPWfzJdWhZH++CliOWaiu/3b4j2kdGmB9urVK3N7q6HO7eU5LRXHjsd5hoUt7f61ToKUQgIAsrXr6u4LQIrUrJ5fFQYfCiifWt9c+qYqcPUcC0z7vb+SUENuMRzu4Rp/+PqsL1YaAsDj1Vw5e/Xm+adfHd7r59FxG/YP8nXYeyz/ddFU7/aBK4KQDv9+OIsoPPXL7A5rNOzG+4x7kn3++rUbgQHjJabZewRBSEUFbyYHTXIxHQIAwOFsly9fLn/1UZMDVq5deuGffwKm+Y0aaokgpAtnL+Gcx2z4Zp2ChqiG6pJqGoU/btw49KeipyOCQWNyORwel9sKgMQ46lhs3J9xl4wt3Y+fPTzeeWhdSca2LQet3a3/2Ly5oxCXWZJH+f3SUalVSrKzv1q7FdiNvpBw2kFfHwDQY4pECxwO/XVGgbXLeAsdHQAAt6Xq1x9+x7sHRn+33lyJs+0WQmEK6CpydJJAlccZICig/KK8Ypa2zfgJHWEwEQRxstC9fS1p6/avnIwMAQCamvbzFs1JSigQ3b9R0LU62dnZYOU8AMDzpGvXE66/yCkbYudx7WbyJJ+JI61N60tep79Oyy/INzU1LUh98N12w8mTfD6aPU2u0R4AAAAWlVmWT167d4uy36C0P934nZvbjbO/bBU4uPILaq18Jp2+9LP1O1rB9jo3j6lnqugqaXEaSvJepqfcuZHIMzEhEur+d/jE6OHDJ06fIpo/tzS3crY0pTVWFuS8scdSvv1qRxmFWva8TKKdsQGhHtbmXauMdx5qaWfuMc5Ny8bb8O0kfI8pEi1UFVY2kprHfYA5cvKyUUtZThVm7uLowInyT224uvt2HbCXHvBzLdwhWrhTesAvJA4AEBne+fNdaUm+HK8KgxKV7wR4k/l41fpvu6ZXFaXOmhTyIr9U5UcUcePc0VXrv6sXCHou+t4jFBI3R0bMX7We2vfHSrgQ97KwrK+rdNfC+eN7R478oDetEWJ9u6zWI8RKeS/qskUHLaWWO3dUv0IjN/e1t/fYrulD3SZ+vjbkzKmz8q3HVBgKqSAxuXjbzq/7YdlJP8Ck0AszCYHTAvs6sncLlVBYp+U9UoGPF0pU6a4FBCFlpGVaOtjYOdn2XK0bXEIX+YKU+ETxaXuXjS/ad+S14xtLQCSX6pUmxqd0HQeoCaq+lVB+3PDF05w33eTS/j78e8zR86o+KNLKKNu5ZvOTjO6OO2C48ffRyA07iTze/ZvnZs1fVdPW4576XkEkBBqy+AAAEslJREFUZP0ee1bcXUJfVJHRAo1cMMPbe82WXUq3hiCIss/upEggZWW/mqBi5XM4VRGzV1W2tHRfpCXxyskDJ690X0Bh2PTS37cfKG1mqLDNd8WRPT/M/vhzYkPW+iVRj9MH/I2sJ9qunzvi7Oy8+8DJ3raEPuAV0b4aj/QRBEFUHEW7LP/5n+efHv59u+xireUNuk5dl94qiYBXy9a0ewef3foAoZD4/O5zMtZ2rPc4J8tuFzsPDlLv32eI/RwVGNgbx2fJUZiQOLH9+bIpPeDnGp0ib+lBiMqUX5WfRtNzLH5yWcvRJzzQp+cKEIiKQbUvh/jlLjiIUZnyk66eouHN6opYW9cvVUmDEAik71DxaB8CgQwIBsMHMAgEoihQ+RCIOgKVD4GoI1D5EIg6ApUPgagjUPkQiDoClQ+BqCNQ+RCIOgKVD4GoI1D54jQ8vHXx2Nmb79oMGQjodcUZz5LqOjsIhkAUReEYO72HUltw4kzqqm2fYCoKHj59EH/6eh2HAwBuSvBc/0lTwz+ciCCku//cu5t491VhhYa23cIVC9d89jG6bY3TVrVx2eaAFeuWhk0TNVhVkHr5SfHadStkRfmRCYKQ7l2/d/nS5ec5pRu3/y6ek/roVvyVa8DJmZNV7TAtcPnnS3qzn6w33Dp9Kq8y68KFu87jA0/6q2anCYKQkq8m33tF3rP/2x5dHQu4tacPnUnNTCXU8CxcXWfMmLlsyUfdBlSDvOf0867gVw9ubtoWI+7XITfl3mRXVw/PoEJyk3jJs0f3BIZFvK6qF0/kcquXB394JfEpgiAIQhe53aovz9mx7VBVC1tRe4S82tuX/loaFujs7Dxp+tyDp67WtXHfZvLPHYuZGvxxTiURQRCBoC7mx81LIzfVcTiKHkVVNJPyg7y8fthztPdNCYXExCsnF4UEODs7L/xkQ48OQNpY5ZGLwmcv+XzT9ujPl4Z5Ozs7Ozt/9f2ed3YtIL2jX5X/JvPJJ5HbiTyeeCKfXxu1JNzZefy5G/dFiQWpDxcv+FIk+8L0R/6T59+4/wpBGn+L3lpEbqKTi6Ii5nzx9c+iKiVZT7/85rdGuY3ht1ZdP3d0YbA/qvlDp68ROZ0Me5P52G/48NgTHU5EiOWZgV5eu/bFKXbaquP1y3/HDJ+S8Di1903VpBfSEaSiIMXPzU0O5bcd3fPj0XM3RL9zUu4GenkNH+6f+F9W742B9D/9955PJxft//F/i1cttNbo9IqBw9mGzg4BgHr79h00YC2pPHv3DyfWf79h1NB27x0OTvaLw9wTDh2KTy5yGuaEac7b9fVvGMtxMwJnidpx9Q4YZcm7eOpqj5bwmFXxpw4uCPt46869dQKLr77fcz0xfv2q+dZa4oZxEu8kUXC2AUF+oiQrR+sxPp4J126/aWzuxZVQnqysLIyJscdYt943ZTd+pBEAeoa6WnJ4FuZw6uvbrD5dFi5K8Zo8K2L1QoGAWFqinkGoBzz9p/zzpy+22g+bPllKPGb/IF9nY+O8l7nFReVsGuGPHfsXbdmIRt1B0cO7rPv2p9MJ532GalXVVaa/4f9w+szRfd+Hz5og3s6c+bOTTpzLrSZ1ZwOHVnnh+L75YQu3/XqwEWvz1fd7riVeWbdyXmfNAwAAo6niwZ0neBszU8uOqBUYjLWLizOd3JSXUajkVegFQiExIz3TyX2YhZHSExpKUk8gT58bJHGHGD58BABA8x25JO+e5CgMBoPpGkJP+YKDEuVm+Pgv79++du2RkZeHsCzbxG3a0k8XyfZ4y6IR7t1+MH31JqmTY6g3/r8u3r6b+G8yMdstZIEo6s5bqOlPXmSkp9cD6wluI/H69O3f/TLW29vP33eEdYdzWryNmZUl7+7t+17rVkgcgt1UFn/l2pVLVwj1DCsX780/bpu7dLaMuTpaM53FYpkMM9bV7TSHZWdnBwA5OztnYbC/jPPtC+hk6pscwozPZxsCcOnwoTeNTQAAAEzH+fqGzerbkJsOnhMcuiSyWCws1tphWNccFdBGq7hw+mImERltw0XwY/nl2XUck427v5QYMHYlOUq6t512NzwAAFFu8HGE4O7nGh0SFa6GvnmUUH7jwd/+TMokHTiyf4TFkGZS3qLg5fG3L/qPnbZn51fd1XnxMKWkSWuzd3fBTPTCwsJuXU48H3dsWeTaNcvmSGQ/S7q/esOuiKgvt29dc2nvXs+F89camXy1dsvduwGnTh0QheLQ0LBw8xxxL+lexGeLRVG9mfWEK5evxF+6Vt7MsnL13rImInyJLM2jMGhMJpdroasjEcNZT08PAEClUmVX7wsIhWXEVpOx48YAAJasX/33/iNWE2d+OMVLIKg7d+6cPC2YWnuGTFdZSOyqqqrRfmPG+UoZxPWSnBd3f9we98mOb4585QMAuHP+1KE7dyYsXN2j7FHd+8YSOgkZdbkntbzLxjOx8eqpfYWV//DWvydOPvt2/88jLIYAAHQNdDU1NMZ7TJIhewBARUUFFoszwnc7TB3u7erh7UbOonh4uHfN9Q/2v3zmqqefO05Q+yo11dwnMMQ/6NzVv5u1hnaOwKNlaWlZX5HbUEe2c7IHADCaivbG/JmU8IgGgPMYv3VfrP8ocII8bzhUCpUJgB4qdDHQcO5UKpULgIyQvn1BalqqoRXe1dMJAObNU/HjF67ytG+Pe5OSeFPOSJuqMoZCKki6kRb1626VRzfIfpa0ecPeyN0/isZ9Agyvgqn3pW+Pzh1LD+yOAyByR6eImclRqOy7cbrnsnFHZHRI3O4DW4O7BNoc1CiqfF5mRiairzvebwz6u7mewmazjXp482QQiUQMFqOr160z2bRH6VVYIYLU37mTODvYX6IcBmM93s8aANBUTykllBJKSoH/WEtnr67ue/X19Pl8fgOxETjZAwCMzEb+tC9uxYoXVy5fvnPt7qbI9AtTZy6NWPpR4ETZD3286RBjANit7DYAxI1ht7IBAObmZv0se4GgNis9Z7i7p60u9eSexFkbV4kGNTic7dHL//SvOa3X42/MXv9FSIDKRhAotIbC33+N8QyZufCjqaJjlZSUDLEy8+xxXrM0MT4F+MZu7fy83x0HgG8s4UW3ug7eGusbFx0dk7xRrR77Ct+wBUKBsfkQnXYNC27+k8AwcJg5a7qMKkIhq4FE1tbT0dKRrpeK/JdH/nr8667tTkZGmc/Si6qI3TVlZjv62uUba7sPsaqvry8UNtYTG8QTXbz8tv92KP7WxaiI4NrnyZsjVy5ZvfHmw1R+9zYbGhvqaWlx27gtndNbWloAAObmFt1X7RPI1eTKkupR40dujdpajUHslAoxrir+vXG1AWPz2cehKm/5yoWredWYFasjRPdlemPFw6Sn7l5uDj2F00aFvyjURTJNchQgiUvoIl8A4m6q10yfosrXXLYsYrg2yHiRDTikpPgzd14Rd/6ya6rE/Rhh1hGYHcfA6piZm3LYbVwOt2uLNFLBrz/ERe3YMG74qMCQgLbWqgdJj2RYYOzmKON5y+VyAcCbmBh3zbIfOWHrrn3Xk69++Ul4fdr9LVErl3yy4eaDVJ60dizsLMwt8NRmemvndbIkEgkAvJOT8oGllCM/t6CmVZfRyObz+PeTH1SyWP1sgIi8V/dflLI3bVit8pZZNELSjXujJo92eRumHQCQdPNeKZU9ZYo/pofa0oSPRsr1dQcH/DAius7lo9LPV6/vkwq/59u7OYZHTK+orbl+vRHv4nfuxieioNooZemvrty8iB8ZuMZV9PkXb21jjQgrWlvagFmn1njcmphdB0O/WIN+w5sdNvtWfNK95H+Xrl5oq9RjraWFpaGhZWHTbfxlS6fRG7ePXrFy5eVLl65evr7lv3vn/Wcui1gWMt1H/Iaio2/vFzj55OXMZjLFzt4aTUQQEqGYYGpr5xc0UQnbegE7LTXNatiwzzd8Upiol/zt/nsJj6IiwtA8gaBufcQGOd/zl4UH9caOkqxnlx+VfL1tnfgHDyoJGWLdkzDlgFxHbm6k+H/oJPpaU1+ZXcZq1NKyGuvT4zwioTAFgMjOMbJLS/IBACnRIeLze3EhGCDxzu8y3BOAuEICAOrzqq+w8s/+7++4G+lHTx8eZS89SI7zBFfuWaH3xE5d5eDgIBT+x6AwgENH4EShkLh31z6HgI9Eczkjxo5w9xrxLLcyOzXPNkCZL1WslhYtHW0LazPZxYbYua3dumvZimXxV+KvXrq6Zc2981NmRUREhMyY9PZ+ozFzxsyEi0nPH730WjkPTWqqJeek5k8JWzrMsF8j+vB4zbmZ+R6jA4bq6prN8vc6cvL6tesfLZhlq009f/zlks8m+oaGS4kc2wW8Ab7nQgAAAATcmhd335j9v71zDWrqiOL4OragKDO0KeUxFkZuEiBQhaARCT5QixKkVXn4QiE+otagErUKtdYpQqsytdhUWoZKiYO8QdomWtRqLaAIGoqJSEJ4CREkgjwKqBD6IRRDcoGboO2Yu79ve7K7s8nMuWfvnn/2zJxNsX2RNK0V30zil4d/xlbfUCkelKddlrKZ/mhDOkqu/jFg4UKjvIfW1LS0NCla+vqMjAafwN3t0uQU4ZP6+ulOdtY2VhhXjoba4Z7qnF/rQI9EwfL7GRa6iv4Svz6KIAjRcR4zLDLrYqFCq0Nvb21oEFtD3N7WLPKmUrk/ZQ9ZlEp5XHREQGiYRjW8+BOfIwjCDIscpTTfSCiV8l3M1SzOYZ3KaHcrqnnxx5cvmIUgxFUbd+ZdKh36KonfHPXyW6cSESuV8pNHDyz1H71q4CtBdrdgvr3zj+l8VTM18SSCOB6P513MTbotaxjPzNLy63QyecX67Rq1urnHDiEIQvXwK6mqU1kaKkvWLlvL0YC9dfkCf5H8EeqQyjvX6Pb29MWBtV1d2k1ti1xWutDZ2ct33e93xH9eSuewY0TSwiUuLtjk0gKWdnVMVem84VZVAT60mtv4qq2pW8xvbbinaDFOSj0lulacn59/QJDu4ukTExtFfudFZKgRVU+2fde0ozoxNbPDmPjhSj+i+VSVVkcoFIKQVQCAAkFWTl5OYZnsrWlOWecvuM+hOVoRmiTlJeW3RGIRgUAQF1+O+NR0rvuc5X5eYxzsqNHV1ikTPdoRu0+n04vJhOkbtu9fvT7ofGZealpa3YOFALgBAAAw3rw7wsEhlxe9v9+W1CdusJzjnpQaZfWfq9bKhH/1EUgz3AbVEIGhgbU1db+cFxD37qbqW3y6WXL3RknRr7n852Zmcmnjd9zEGWQybYmnKktnYW6JWBCetNSKy+7PQmyaa8oOhh+StbbJCmQa81Dn+zpZmWsPAQBYTDN3cnMwsnY1nTJFu6ltsbJz232AnZmRHnX4iNci7/BYjiQ/u77blOY+G2CjCMOGnUTxAAA1u+9Mxs9eH+gS86vLb7A/jhoK5kqlPOfsabq9fXTcGfVuyaeP8XIuyaW3rxSL1e11FcVL3RmFoqrxP65GIvdsfCg7oqlfp5A/jL6eut9+LnmJS3qtyUtJuHFP9qqHjMzjwxzWXC9/bJssAQtoV8ZFCfCoRXdRBxs22KPj03PnUhxoFOt/X8MmTLBaGbyZ4e/d2fniGH9g4GHFLZnJGzV3FRMX0YZpcmwcaFt3MJLP8LrH+7BCp/WhmH+hMvLIJ+PRlkycZOPt92qVsK8Lf7dJ7zUauTrqkMXQY8hos7U/Li0Qvu/qZKspp0KFREE5n/dZwQIAFO0JiRu0q8S9gLViWOq+SiICwINCeinrfk3A7iTGJlNMJBKJukmpVLTJO92o1CFLZ2t7Y+9Uo2f92Zk52lP4bwz2sDM5/X2K/usdgd7O6m+/SArmbCcRzMbuDRmLh1VCLu9mKCcYe35FjyGjI75TUf/4GZ3uia070TfIAxRl8DVc/weByvdJqpQeIwEATbXPYNYfZ5t9HSpqNkpKI8O+nMfcxFj2gbVZ74P7kqzUbBOix7YNfkN9bl/np12tjIkM3PrRfvapEwNSEdnHZ3huvVuQkVbVNXXXphHVOLrS0yE7dZwfwGEib2seuXcoKsJC9jX392OfzXf1ljBmwMtaG0Q/nrbXRB2Jzr4o4/K4i2c7YRpTFUcn7QEocj114T6KnE/1Md5KautaS7dJeLWsvkXRM8nS1taWNJOkkT3jfnXI0nVZwFLP64KM2OQrew9yFrjaa8/SU9082Q49KagH/c8but+chppn61BUbAnaqdOtdWu2hEPP/38pysur7+lRMxj5rlmFIY+qnwtf2DaBkTCqvtcggVW0IQaE6r+4uvg+PgM+gHfvQgwK1Wt9AgPrbRtVcSF7igBLgDe3BzDmQwyOwTs4xg7jmDsaJNDzIRA8Anf7EAgegZ4PgeAR6PkQCB6Bng+B4BHo+RAIHoGeD4HgEej5EAgegZ4PgeCRfwAk6Cql0htwiAAAAABJRU5ErkJggg=="
    },
    "feeb42bc-f7fb-4eea-885d-40feb008a458.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAADzCAIAAAC0QmJ5AAAgAElEQVR4nOydZ0BTVxvHTxL2NuwlyFAEFHGhIFjBUaAi7iqu2ha0WqWODrWttnaJWhyvgzrqVrQqKqB1W0XZIEsIeyQQIJtA5n0/XAwhhJCEgELO7xM56z73Hv73nnvuOc+DQRAEQCDvHurOjTsCV28I8BrRY1E2s2xB8FeHks47Gxn1g2WDjMxniZcfF2PftRkQCAAAcLnM0gq2g6t9jyU55Or/HvzXygO5aXnVzbx+sG2Q8TLlpc+kiRj4zIe8D5TlP//z/NPDv29/14aoCxrv2gCIulOVn0bTcyxOy5o+c/q7tkWNgMqHvGMKivJp+Po6ttnWQJ93bYsaAUf7EIg6Amf4IBB1BCofAlFHoPIhEHUEKh8CUUeg8iEQdQQqHwJRR6DyIRB1pH9W8jT8FfvX0zdUKwy5nGUQNid84YJZ+gAAAHi8mgd3nj1/+vjByzwAgNnQUfMXzJ36YYCzsWG/GAaBqCn9vZKH2VT87eZtNuNCt29YLUosTH+4akkUx8jl/O0zo2wt+9MeCEQ96e/RvqHZiLVfrE04de11bb0osbKiigKAt6+3O5Q9BNIvvIP3fJdRzhY6lFfP0t8m0FPTUgHATJrkg1O2zdTU1KKiItXYB4GoAe9A+Rpamsamhg0NDehPDof2Or1AS8vOe7yX0m0uXbr0559/VpGBEMjgp39m+Bof3b6fkHC7CWdrxuEFfBpCb2ZQKBQ0r6qwsoFEth4x0d7Jtl+MgUAgfa98bmvVz9t3vyxv2737t0mejnxezRcLo+ubmn3weLRAWlo6WSicM3aMjZZWXxsDgUBQ+na0jyCkvbv33Xpa88POHyZ5OgIANDQtHD1saQCYm5sDAIRCYnpqBgDmPj4T+9QSCAQiTt8qP/Np5o0rSdOCpwWMcUVTEITS2NCEwVhZWloAABhN1KKcEgMj49ETPPrUEggEIk6fKr/5zp07LE27ufPniJL4fH5dNcnMzsp78igAQHF+aS2D4eg+zNzctC8tgUAgnehD5fP5rYSiMmNLU8cRQ0WJlfkVxGqSj98ERwMDAEDqq1QuABMnTpSm+9aSOmbfmQeBqDN9qHweh0ttpJtZ4HV1dUVp9x88aEQsAoMCAQB8fm1WejYOZzPBZ1zX6hUFOc/+e9Z35kEg6kwfKl9TW8vcGq+hidPCYNCUyoLMa+duBoZNnz5tAgCgvoJUSagxGGLs7D5Moi6fV3PxxDWfKVLuCBAIpPf0ofI1NOxC5gQ31VPobDYAoKk678fvd7oEzd2++xs9AAAAGamZtW1tOvo6OIy4GcK8Vw+2b9qe2dxma2fVd+ZBIOpM337PX/TJohYme/uKr2ysW+maLvPW7QoO8tECgNlc8u+9e5dO/GNqaspnVS+ascTW3sBAByvgcRiNRDKTDwBYtmYrvk+Ng0DUmEHiddvZ2dnHx+fixYvv2hAIZGAAI230P0jqo1vxV64BJ2dOVrXDtMDlny+xwim9WQkCUQao/H5GcP74nycSMg8c3e/lYC0UEvf/tH/zF9kxh36Di5ch/Qn0xtWvFGf9d2z/X3Pnz/VysAYAYLE2ESuX1qc+PHH4zLs2DaJeQOX3J5zEO0kUnG1AkJ8oycrReoyPZ8K1228am9+hZRB1Ayq//2A0VTy48wRvY2Zq2fHVAoOxdnFxppOb8jIK36Ftg4LkKAwGg8FEJaus4CAGKr//oDXTWSyWiamx2KJGAACws7MDgJydnfOuDBscJEeFxAEQmYQcD5bMkFR58HGEEOsL4kLUWPtwhk95BIK6ixcfyVPS1NozZLo3g8ZkcrkWujr6nXP19PQAAFQqtQ9sVBuSo0LigG8sQVL2oPTA7jgp5V02nomNd40OiQrvcqdQD6Dye0VK4s2sitoei3353a8AACqFygRADxW6GPr6+gAAKpXKBQDO7ysFKu/IHRtdxBKTozAh0kTfjsvGHZHRIXG7D2wN7lRNTYDKVx4czvbo5X/kL483HWIMALuV3QaAjlg6u5UNADA3N4OyV5LSxPgU4Bu7VcGHd/DWWN+46OiY5I1q+NhXjfJfvXqlknbkwc7Ozs7Ort8Op0IMjQ31tLS4bdyWzspvaWkBAJibW7wrwwY67cIPlXhyBx9HkOMAlB7wc41OkVrRJXSRb3RK3M3k48FqJ33VKD8iIkIl7cjDhg0bNm7c2G+HUyEWdhbmFvjGZnorhwO0tUXpJBIJALyTk9M7tG0g043w5QGVfn5JKQhWtwG/apRfVlamknYGFgJB3fqIDXK+5y8LD9LRt/cLnHzycmYzmWJnb41mIQiJUEwwtbXzC4KeCJWDUJgCQORwpaTrMtwTgLhCAgBQ+RAF8A0N95WjGN4A/YCvMXPGzISLSc8fvfRaOQ/Naqol56TmTwlbOswQhhJ8B7i6y9OBgxCofOXB4WyXL1+uUJVRkwNWrl164Z9/Aqb5jRpqiSCkC2cv4ZzHbPhmXR8ZqSb4urv2orY6Dveh8vsZ7U83fufmduPsL1sFDq78glorn0mnL/1sran5rg0b2KT0asDuqdyrwoAGKr//wfrNmu83a/67NgMCQPskgee7tuIdAFfvQgY6ru6+6IBdCUpL8nv7qjBAgcqHDHRcQhf5gpT4REnpt+/LQT/mx4VI26RTmhifop6D/f4Y7Tf8FfvX0zdUKwy5nGUQNid84YJZ6ML1xvL8h08fxJ++XsfhAICbEjzXf9LU8A8nIgjp7j/37ibefVVYoaFtt3DFwjWffazTw1Eg6otL6CLf6Oj4xNKNCi7DRYUfGa52y3gAAADpRxiNb75YMW/3gZPiibkp9ya7unp4BhWSm8TTzx7dExgW8bqqXp6WnZyclixZokpbIQMJQqwvACAySbFaSZEAAN9YQt/Y9J7Tr6N9Q7MRa79Ym3Dq2uvaelGix0SP0eM92lrpmS+yRYmFaY8SH9bEHto3aqhlf1oIGZi4bNwRCRTcdSt1l4/60N/v+S6jnC10KK+epYtScDjb0NkhAFBv376D7lMllWfv/uHE+u83QNlD5CX4eJJC2i89sDI6BUQmqeFeHZT+Vr6GlqaxqWFDQ4N4on+Qr7Oxcd7L3OKicjaN8MeO/Yu2bJwyWj3vxRAleat9+XzyuKq37vtlhq/x0e37CQm3m3C2ZhxewKch9GYGhUIRL2FiMSwwNOCvi7fvJv6bTMx2C1kQPt2n7w2DDDLaN+epsOAgpm+Vz22t+nn77pflbbt3/zbJ05HPq/liYXR9U7MPXiJ8jl5YWNity4nn444ti1y7Ztkc6c1BIBAV0YejfQQh7d2979bTmh92/jDJ0xEAoKFp4ehhSwPA3NxcovBwb1cPbzcMYuDh4d53JkEgEJQ+VH7m08wbV5KmBU8LGNO+RApBKI0NTRiMlaWlpBeKtEfpVVghgtTfuZPY1nc2QSAQAEBfKr/5zp07LE27ufM7hu58Pr+ummRmZ+U9eZR40Yr8l0f+evzrru1ORkaZz9KLqoh9ZhUEAgGg75TP57cSisqMLU0dRwwVJVbmVxCrST5+ExwNDESJNFLBrz/ERe3YMG74qMCQgLbWqgdJcjm0lQNK1rOk+VOnfvzpV9nFDQCwctOTQ/3nnrvxgK9sFS6n6tOlX5F4PBVZCIG8G/pK+TwOl9pIN7PAi/mW591/8KARsQgMCuwoxq2J2XUw9Is1U0a7AGAwO2y2BRZ7L/nfOg5HFVbgxwaERHy2mE5n2A63AMCgJL/sq727l8+d3v3EZrdVEF7N1qjt+VRk1KjRvNKs6HW/ULptBAJ53+kr5Wtqa5lb4zU0cVoYDJpSWZB57dzNwLDp06dNQFOEQuLeXfscAj4SfcMbMXaEu9eI6jeV2al5qrJk2oypSFVlYc6b/5KvCsxHTffxUK6Kpqb9z79+Vn73jokFKbWGs+WPr/E9NgSBvK/0lfI1NOxC5gQ31VPobDYAoKk678fvd7oEzd2++xvU2zyCkA7/fjiLKFyydLZ4rfE+44RC4vVrN9iKHM7Hx2fkyJFSs4ZYDZsSNPLStYsZtYKPP5oqT2tSqwiFxAvnL5a1GGL57qSX96+ev6GIgRDI+wUGQZA+ahpBSKcP/v3ov0ob61a6pktI6KzgIB/UpfzzpGvXE66/yCkbYuexePnCST4TR1qb1pe8Tn+ddi/xbnphDQBg0vS5kyf5fDR7moHsw8hBQeqDqC3Hzif/jc4vFGY82vx9TNdi7hOC9v20RWoVAAAApBcPqT5TjXZ+c/H7mGWpj6gB03sePkAg7yd9qPz3BC6nKuanhNLCfz9YtmHl/Jm9rCIQ1F0+VxCxSq52IJD3lkGufKGQeGDHufnfb0iLP3nxQeHpc0eMe3rmS60CgQw23vEu4b6FfiZ2b241CUGQZmLezAkzXhWV90EVCGTgMWi9cbHqy47s2fu8iqEj1AeguaG+wVCPe+TQsZd5VSqsAoEMUAb5aB8CgUhl0D7zIRCIDKDyIRB1BCofAlFHoPIhEHUEKh8CUUeg8iEQdQQqHwJRR6DyIRB1BCofAlFHoPIhEHXkvVI+c++uLfMjviRyue/aEghkkPMeKb+xuvxRwn+zQmfaaGm9a1sgkEGO8spvqip4nP5GhabcS7rPMHWYGTZNhW1C1IPkKAwGg5Evop58BQc9Siq/Iv/l6sWR61au/e91iUrsaGOVJyUkB3/0obhDbjGQ1EcJm6OWb/5j5/rFq2OOna8XCFRyXMggIDkqJA6AyCTkbYDM0gN+GDHEdB58HCHE+ioYb3twoty2/vMxBwlU2tXThw6cjFeJn4D7N86NnRCSVVErLZN/7ljM1OCPcyqJCIIIBHUxP25eGrmpjsNRyaEhA5ukSACAbyyhc4IEkUliNQixvpJJ6ocyz/xWVrnOqHEuJsbjJ44jElUQD0cgqEtISPCbPmWMo23X3OKs/47t/2vu/LleDtYAACzWJmLl0vrUhycOn+n9oSEDnNIDu+MAiNyxsVPMdTFVo/eBuN0HSkW5Lht3REokqR/KKF/XwGn+LF8AgL6Rnq6OTu+NKEovyshsDgsPw0jJ5CTeSaLgbAOC/ERJVo7WY3w8E67dftPY3PujQwYwpYnxKcA3dmuwWFrwcdGwHwAQfDwpEgCQUkgQL7I11hekRMeo8ZC/V3P7fC5fpyOEjtLU37hxw2Gi1+gJUqLoMpoqHtx5grcxM7XsCGyBwVi7uDjTyU15GYW9PjpkAIMKf1GoS89FO+MSusgXgLib6iv97sNMyUHaywx9fTN5Sp4/FnP+5sPucpkVFWwHhxXB4jdu4DzK/38x22nNdBaLZTLMWLfzLcbOzg4AcnZ2zsJgf+WMhwx82p/4soWffDMOABAZ3um/C7iELvKNTskvKQXBCt82BgXKK7+y4OWhmGNLt/0qZ3kmkyk1XUgmNxpaWLa0SGTzBFwAAIPGZHK5Fro6+p1z9fT0AABUKlVRsyGDCEJhCgCRw2VJF50HkHgfAAAAl+GeAMQVEgCAylcAPr825o+9DRYm+vp68pRftmbrsjVbu6Yzm98sn/fZ/CVrt6yJkFqRSqEyAdBDhS6Gvr4+AIBKpXIBgOt+IN1QemBldArwjT2zsau8Xd1934FF7w3KvedzTh86/TiD8tXKj3T1e/We//T+82rOkA/DZnRXAG86xBgAdiu7rXM6u5UNADA3N4OyV3N83V27ySk94OcanQIik15I0X07+SXqOr+vjPIznt4/E3cxYs1yK80hurrKz+3z+bW3b93+YGaAp41Fd2UMjQ31tLS4bdyWzuktLS0AAHPzbitC1ITOk/YdJEehshef5++Kp8xXhcGMwspnNBbt/W2/0SjflVERdBpdR0955ee9zMvOY4XODpFRxsLOwtwCT22mt3I44ukkEgkAvJOTk9JHhwxeSg/4YULiepA9oTClH01671DsPR9BSIf2H3ldjRw8/ZWdtjab1aqtow0AQBh0jFFH+LmEC8fvZZOP7P1elCJ1bl/Ir6NxjGO+3ywlxt3buX0dfXu/wMknL2c2kyl29tYiMwjFBFNbO7+giQrZDxlcuLr7ogN28fl50SBf9tO+tCRf1qvCoEexZ37y1eQrV5OXfB4x3ccTANDSwmIxWQJB3V/XnokXCwwMXPnJSom6TEmqyysF5uZIl/R20Ll9ADRmzphpJmx4/uilqKmmWnJOav6UDyYPMzQEAAi4Nc9u3S+soilzASADGJfQRb4gJT6x07t6+7M8LgTTGb9OS/ZKE+NT1Hmwr8i6/ZritJljvWfNX1Xd2oqm7P/56zXf/hh3cOeDtHwFVw1zD/+xY3rYclFTMmk7Ebt72uylr6vqEQQRCol/7v5m1vxVlS0taPbhP3Y4OzuP9Z2dXlqloBmQAQ4h1rfzon3py/YBkCgFl+7L+8zn82v/jDnA1B66eesm+7crdqcGTK3LILTqegRN8EBTeLya8yf3L5g250WB9HkXFFpDyb1bD2fP+chersW/2p9u/O7ndfPO/rJ1068/b1y2k2044vSlOIe3X/osza2cLU01GisLclS5axgyAECf+p2W4QYfl/6vLj7DnxwT3WXRr7qh8ntJC6N0yezVsh/m184cnjBlTlFDkwqPm3Ah7mVhmQobhAwM0Ge8Io9v+MBH+iKKdg2hRtvBQsbDnMeruZOQGPRhoJuFqaoO2kIlFNZpeY+EU/3qB7olR/4d9+jqnsgkmfN/gx/VKz8397W391gZBTKeZOQWC0LnqOzCk0qzD599tWrTMm1VtQgZULzVvnw+eVyh7gEAAIMgiEobpO7cuCNw9YYArxFSs4VC4jfrvmnE2R04/Jux1BIQCKTvUfEzn8tlllawHVztuytQnleW/qIqPDwcyh4CeYeoWPk1JdUm7i4OkvtrRDATbt7Sdxs+McBbtceFQCAKobLRflV+Gk3PsfjJZS1Hn/BAn25KcRmMNgAMjIzeI2/fEIga0ivPHOIUFOXT8PV1bLOt3coeAKBlZAQ310Eg7x6Vz/BBIJABABx1QyDqCFQ+BKKOQOVDIOoIVD4Eoo5A5UMg6ghUPgSijkDlQyDqCFQ+BKKOQOVDIOoIVD4Eoo70k/IZTUW79sW9qYPucSGQ94J+WrfPaCpaOGtZDTJk3qJ5S5Ys9nBQ1A8XkvroVvyVa8DJmZNV7TAtcPnnS6xwuD6xFQJRB/rL4R8v+7/kLWuXjXR2dhs17Zuf/8wm1Mtdl3/uWMzU4I9zKokIgggEdTE/bl4auamOw+k7cyEDjXZn2z371ZS74OCm35SPws95fvfrL1aMdHYePjJg0w970gpreqzzJvOx3/DhsSeuiFKI5ZmBXl679sX1pamQgUQXB7yof10RnV3yQ+e7/a58FEHOi7vfrFvp7uzs4uL35Xe/pLwu775w275dW0eOnJZVUStKEgqJmyIjxk6eXURWpd9uyEAlKVJC3FKjbYgrHWr/ncztY718Z/1++PTlc0fnf+jyKP7UinnL127d9SxLSnAORlPFgztP8DZmppZ4USIGY+3i4kwnN+VlFPaj2ZD3k9IDu+MAiNzROVZ21+d/3M0O17wuG3dEAhC3+4C6BtFWcG7/0c3L/4v5IfrzJZt2dETBLEh7OG3atAsJjxQ8NGbU5Bm/H/r78vljC4KHP79xbvWiZZ9Ff/8wtUC8EK2ZzmKxTEyNdXV1xdPt7OwAIGdn5yh4UMigozQxvku8nODj4uE0XTaeifUFaOzNjiJbYyWj86gViik/MPzjaR9MTXlc4OXt9TaNd/fu3YYGjOfYkcpZ4Dlp+m8HT12+cGxRiEfanUtREStWrf/27vNcNJdBYzK5XB1dHf3OtfT09AAAVCpVuYNCBg2o8BeFygyNSShMAZKFXEIX+XYeCKgVCo/2i4resLTMPL3bdc5sLn+c/NTB3dHSxqI3dnj4TP/6668joz4GgP787rU9e/acjX8AAKBSqEwA9PT09AAQCoWi8vr6+gAAKpXK7c1RIapGvI/6BXmEnxwVIuV1oF36nQYCaoSiym/KysyydbW3sbdCfz+6++RNI8V7rHdvvq5Ta4oO//H9nJB5uSW1I8d/8P0fRy7furBi0XQAAN50iDEA7FZ2GwC1tZUCgQCtwm5lAwDMzc2gQ8/3CvE+6hcIhTKCYSdHYTAYDCYEJCEI0jWqjstwTwBSCmXFfh28KOZ7l9PGys8sGhU411pTEwDAbH6TRarEYKwmTBjfqRzCrCsFtq6GPTZIqS68cPHitQvX61q5E6d9FDw3JChkpngQDkNjQz0tLW4bl8Zp09DQxOFwAACBgN/K4QAAzM27HWgIBHwKpUkg4FtZ2Sl0ju8JbHYLDofT1pYn1vD7Akesj3qDUCggkWptbR1UYhUAIC4EEwd8YwkvJJ76ru6+3dQY/Cim/KqiquaGtkXjxgEAAOAnX3yiR2/TNzbwHOcmKlOW/urKzYv4kYFrXMNlNNVcVXDh4sVr528QOcgY/xkbPv7Y2916qLWdhEEWdhbmFvjGZnojsWakvSMAgMVi1tfXsLk8APBOTtJDaDY21iOIsLm50choiEIn+D4gEAgaG0kNDcRhw4YPLOWTySRra+n32YqKktZWtpvbaCy2h2Emm91SX1/L4ynwGufr7io9I/g4ghwHAJQe8HONToleeSBUUvsAHe4Hy5wlGJR0dAOLxZBRDs3Nzs6haJqgL/m5Lx4KPJyz0nId3YaZ4jsE5jzBlcsQek/06q6p5qqCg79uCw9ZePDkHfspwX/+de7K6YMffTBWG4vV0NCUKKyjb+8XOJnHaGlpaUVzDQwM8XgcpZliamvnFzRR6iHMza0sLGwGouwBADgczsrK7r3SvOz/DRQOpw0rrQdRNDW1NDU1MRhMj+3o6embmVkqZF6PA3b0G1435bp7VXgHyHOdVdVyu/KFQmFzM7m7am9z67Ozcoa6OVjYmJMrc+/nNE+xM6qvqffx8THi1x47egMtzOGwqsgYe0fbru00VeYf/OW78OAFB0/ddwmc878z5y4c3zd72ngcAI2N9RYW1tIOrjFzxszJnuY52R0TMfRGWkNd45QPJg8zNAQACLg1z27dL6waVNuB5BFJ/yD7f0NE9z0IAAB2do6urh7vz0mhEApT3rUJHch5nVXVMhYAwOPxamrKu5uVFeXyeLyyNxUmeJPa53cPn3y+bM3imqo6Ig/v6uF06o/roZ+0T6BU5JfrOlgYMspPHPp5f9yl0kYWAIBNLz2Aav70M/fgxScunzt96LdZfqPRKlwuBwCgqSl9tm7EuEmTAibEX/0nr7oBAIAgpAf/PsQMsdzwzTq0wLHYuNWb1i5fuiKjrLqX1wgigez/DRGyexBBEC6X0zcPNFd3KfPzyVGdAmqXHvALiQMARIZ3nuQrLcmX8arQr8h5nVXYsgaN1szlcoVChMNpI5NJAAANDU083gzNFs9tbub6BflnVdCSs0jrf/pCl8XQ0DdY//nMx0kPVqyYj5Dryng8PN4sLS198uTJLBrbyWd+4ER3Pp9XUJCNA+z7N154h69YsmSJ7+hhEkbIflw0NlLmfrzKztLi7C9bBQ6u/IJa12n+s2d5W+GwdXVVAoHAwWVEkN+EnJcFBTlvxjsP7fFacLmchgaipqYWBgM4HI6lpY340Bq9DtraOnw+TygUaGhoNjWRHR1dDAyMemxZKBTU19dhsVgsFtvW1mpqaqGv3zHNyeVympvJOJyGQMCnUpu4XO7o0RNE7708HreurhqDAVhsp4m91lZ2bW2lUChwdHRtaCCi7ZiaWgwZ0vN+R3nqotNpQqEQg8Hw+XwrK1sdnfZFU7L/N8SR0YMMBq2trZVOp2AwWBeXjmsouxcAAG1trfX1dQgi5HK5ZmaWpqbm0s4I67v+m1G/vUws3bjRpSNX1/Ra3FziXxhgYmKckZGTlwcA8D24ZWZNTXlzc6Ox8ZChQ51xFYmFWv67Vjlp11RwLW20tLTl6UGpNstznWX0fo/XuampgcfjoVOnbW2t8h9XVsvo+sbGxvry8uLulvh2l0unU/PyMojE9l03fD6/sDDl963fXLsVd+9lDprI4bTl5KTl5TztbnMOl8upru523b7UXAqlsagot7a2UigUIgjC43Hz8jJvXzn1srBMvFhtbVVtbZVEXR6PW1iY09bWKjKvsDCHy+WiPwUCQWFhDpfbvguwrq6KSKxmMOg8Hrc7C0UIhcLi4nwKpRH9yWazcnPT+Xy+qEBJSb7op0DAf/MmT2QGn88vKMhmMGhv67bk5qbT6VT0J3qda2rKBQIBegqvX6fz+bweTeqxrlAoLCkpoFCaRMfNy8sQnT6K7P8NpKceRGlubiQQCkU/ZfcCanNVVRl6ubhcTm5umqgLJM+o+OD+bZs/OFjcKTfzpL+mBgBAX19v27bNuh8cFK3pLy7Ob24mIwhCiPXV0tLcf7fj1GT3oDw2y+gjGb2P0t11JpFqS0uLRD8ZDFpxcb7opzz/G1Jb7u26fSwWJ5rOxeFwetqaujZDtbiCf65eRxO1tLQ9Pcd6jPafMFL6rC+ZTJLxwO8ul8vlWFnZoe+NGhqaulr8plZd75HSp/rFIZFqjYxMRI8XLS1tI6MhJFIN+rOlhYnD4USjVkNDYzqdamho1N3ElTjNzWQEEQ4Z0n6rxmCwmpqaQmH7x20EQVpb2aIXXSwWZ2fnKKrb1NSgoaFpaNj+QVNXV09bW1u8cSwWZ2vriD4iNDQ0NTW12tpaezSpx7o0WrNQKBQ9InR19fT0DJqaGuRsGUV2D6JIvODL7gXUZnv7YehTTlNTS1NTq62tTfoZDQ+21WeV7z2eLJ7r/ckzLg9BEBarZfnyFY13Volm8fB4MzqdCkByTHSK67yDMyZ27AeR3YPy2NzddZbd+7LR0tI2NOwYK2lr67S1scULKPe/0VvlS3y8baipN7U2+TAslFNUnFFak56cTAcAh8N1N7WDjqi7m8eWkaunZyA6NKk0+/5/ZaMmuGl3LdcFGqTbLMsAACAASURBVK1ZYtxuYGBAp1NEP4XCTq5KsFh5v07T6VQDg461CDo6uiNHeoluIhgMxtgYX1KS39TUIBDwAQD6+gaiU2My6QYGspY/SFxDDAYj/4IZGXUZDJqeXqeF0draOi0tLDlbBj31YHf02AsSNmOxWJECu+S6eI0bqa2dEPL21V72tRoyxJTJZBAO/h4HwMpVM0eKfQCS3YOK2ix+XNm9Lxs83szCwobH41IojfX1tY2N9UhnbzrK/W+oLIo2Sk5Wjr3HOBzOdvX6eT9t3735203ohWQ0FX25cktDF4M8PEZUV9cxmVL+1UIXfzZ/1kRz8x4eJgAAaxfvRXh78btvdwiFQoFAIDERpampJRAIhEIhFos1NDTW1NRkMuno45dCaezxaSaCz+dpasoaGjg4ONPplMbGhrq6KmPjIdbW9qK+Fwj48gwrVA6fzwOA39hYL0pBEET0ni8PZDJJnj4Sp8deUKg1AABwDDw471DwHyFR4UiMXw9lcTgNQ27+jlMUbFTS4pGI+KNLRg/23mYZvS8bPp9XXV2uoaFpbm6Jx5tzuRxFB2VSUbHyl3z2ZXl5CRaLDQhZFBCySDyrtbWVyeGIp2hrawGAEIkkqU1hsYDP58v5XygQ8OVZOobFYrFYnPjTAwDA5/PRGR0AAIIg2to6bDaLzW4BAODx5qIReI+gkzeyyxgb442N8RxOW1MTuaSkwNXVHT1BHE7Sqv4Bh9PQ1NQyN7dSrjqfz5e/j0T02AtK4LvtYOQfoXEhmCHJ1E8kZ5A7kxwVsunJuHGfJcb4GvB44jkyelAlNnfX+7KpqCjR0zO0te156loh2pWPw+HEhxAsFkN8YCMjV8KLH5vN6jpqFQgEhqZu8Y8kt/GSSDUmJqa6unpSLUNzu7Nb4itFW1ur+BysDIyNTdjsFvGzY7NZxsZDRH+zWAwHB2dtbd2uPYrO+ujpGUh9eTEwMGQyJT9cNTU1oOtSGAwaBoNB7yPa2jq2tkMxGECnU9G+19MzlHg34/N7uInIaZVsDAyMaLRmiUQ2u0X8FUBG7zc2kpS7a8juBaXwRdfrMRg0kvRHyVuCjxd+iBQW5hhSm4xtO71sy+7B3tgsu/dRpF5nHo/X0sKysenVQmapLbf/c+vo6KEzjQAAGo2Cw3UaC8jIbWtjiy4WejOTWCfP5XLy87MIBEkXGgKBgMvldid72bkAgNbWFg6nfcqHx+MymTQ5h+XW1vYUSpNocSifz6fRKNbW9uhPPT0DLS2dkpLC16/TCwqyy8qKGhqIorsMkVhNIBR291phZmbF5bbR6e0bhxEEIZFqRP8oWCy2vr5OvAOEQqGengH6t7m5JYvFEM3cNDU1oNOz8pyRbKtkg8eb8/l8KrVJlEImkyRued31fo99JAPZvdDXYDAYY+MhfL7kOFF2D/bGZtm9jyL1OmtoaGhoaLS2toiKieRGoTTKeb5SW8bt3LkTAKCpqSkQ8EmkWhaLoaWlZWRkIl6zu1wOp62trVVLS5tGo9BoFCq12dbWQeKZLxQiVGqjjo6uxPdnMpmIx5t3t/ZDdm5bG9vYGN/SwmQw6HQ6hU6n2dk56ui0/wuinzFaWpg0WjOH04YgwpaWjkciDqdhaGhcX1/HZrew2Swmk2Zr6yB646LRmhFE6OIy0sLCysDAWENDi0ajtLa239rR5ShDhphJzIqhYLFYExN8Y2M9ldrEYjEYDKqJSUdJoVBIItVQqU1cLqetrZXJZGhr65qYtE8s43A4Q0NjIrGGwaAymQxtbW0+n8/jcbW1tQUCQUMDkc1mYbFYdFxDJpNoNAqfz9PT09fQ0JRhVVtbq+y6GAzGxMSUQiE3N5OZTAaDQTM2NpFop7vel91HEqB3atG0uYxekG0zn89XOld8JgVBhNraOrq6nc5Udg8qbbOGhqbs3pdxnTEYjIGBcWNjPZvdwmIxmUy6vr4hh9PW0sI0NDQRCnv435DVspwPFqkwGDQSqWbEiFGKVhQKBTU1FQ4O0hdMy87tU4qKcp2cRohPvTCZjPr6GldXj/435n1Gnj5CEET0AtLQQOTzeSrcftdLiMRqS0vb3m8rHLj09quecjcOBAEyNs/Kzu1TdHX1GYxO6/9bW1uMjfHdlVdbeuwjGo2Sn5+FvigJhQI6naLoPpy+gMvl0OlUdBpPnWUPeqN8Nrulqamhra2VRKpV1BmD7J3n73Bf+tChTgKBgEisbmggNjQQ0Zdn+T/sqQ899pG+vqGurl5FRXFtbSWRWOPg4NLHfdruhSNKpnMtBoNWlXpl+fKlzs6/q6kXLhFdVwtCIAOOLv72xWh3vdvFGa86+9x+R163IRCVkhwVEgd8YwldHW61O+WQSESd8caFyB4gDG6g8iEDHen+9gEAACRHuUan+EZGdnG6Bf3tQ+VDBjjS/O2jGQf8QuJAZNKLre5SqkF/+xDIQKYbt9voKD8ySYrL3Xagv/0BA6W2YM8vp8gIAgBoLM+/fDp2XkCAj4+Pj4/v5p0xN++mAQAQhJR87e+Nn3zs4+PjFzA/9sRldKEfp61qzYIFF289lmizqiD1j/+dVZGzGCT1UcLmqOWb/9i5fvHqmGPn6/vV/7R6Il346Chf6ot/B+rtb3/AzO2/enBz07YYkpijCwRBclPuTXZ19fAMKuwcWvPs0T2BYRGvqzoCdXO51cuDP7yS+BRBEAShC8QK15fn7Nh2qKqF3TsDYajvd0JSJJCcpe8yc9/tVL6UymrDwHjmF2c9PR6fvmVXtEQ8D4+JHqPHe7S10jNfZIsSC9MeJT6siT20b9RQy6KMxwG+C24+SNXU1HUfMWr0BA9G45s1y1Z8+c1uUXnLYV4rFoze+9OBJqA8xVn/Hdv/19z5c70crAEAWKxNxMql9akPTxw+04tWIcqA+tWMC8G8BZ3cjwvBYPw6z+ips7/9AaB8Orlo/4//W7xqobWG5J5iHM42dHYIANTbt++g2yxI5dm7fzix/vsNo4ZaAgAcnOwXh7knHDoUn1zkNMwJ05y36+vfMJbjZgTOEm/H1TtglCXv4qmrytrISbyTRMHZBgR17A63crQe4+OZcO32m0bJzXAQ1dI7J5pqOtwfAMo/f/piq/2w6ZOlO/D3D/J1NjbOe5lbXFTOphH+2LF/0ZaNU0a3v/bp4V3WffvT6YTzPkO1quoq09/wfzh95ui+78NnTZBoZ8782UknzuVWy97hKR0Y6vvdIuFHP/i4xMC2Y7QvJdDG++Rvvz9RWPlCIfHbzz5bHbVdtEWwqe51REjID3uOylGb//L+jc1RX+46cuzHzVF//nW5oSc3wywa4d7tB+PHj+9ujbWJxbDA0AAut+Zu4r9//LTXLWRB+HQfsXxq+pM7R2N+PJWUPdJtpJM+fft3v5yKTyomSTrnx9uYWVny7t6+L8dZSAJDfQ9Q3it/+/2Mwj55GE3UrKysyUsizd+mFOUVpxEYi7aM7alq48Hf/kzKJB04sn+ExZBmUt6i4OXxty/6j522Z+dX3dV58TClpElrs7dn983qhYWF3bqceD7u2LLItWuWzRHPe5Z0f/WGXRFRX27fuubS3r2eC+evNTL5au2Wu3cDTp06IL49WkPDws1zxL2kexGfLbbWaLp4UdKPiFRMrT1Dpnujob4tYKjvd4DI374SAbLeI3/7/Y/Cyq8orqpnGowd7/02QZCammowxMhz3EjZFR/e+vfEyWff7v95hMUQAICuga6mhsZ4j0kyZA8AqKiowGJxRnhZvu6He7t6eLuRsygeHpJLNvyD/S+fuerp544T1L5KTTX3CQzxDzp39e9mraEGks1oWVpa1lfkNtSRrR2wKYk3sypqZZ8RAODL734FnUN9iyMe6hvG/O0bXEIX+UZHx6P+9hWjNDE+BYBI9RzsK678jMxMrLGe+5gR6M8Wevl/D1KGuXmbGcsORMHLzMhE9HXH+41BfzfXU9hstpGR7FoMIpGIwWJ09WRt80p7lF6FFSJI/Z07ibOD/cWLYjDW4/2sAQBN9ZRSQimhpBT4j7V09pK6X1Qfde1AbMQ5jT16+R+ZhnVCPNS3+NFhqO9+QGnptws/XNYn/0GMYu/5QiExNzPXYYQjfki7Y5Ynd58VkMg+Pj49+qkUCAXG5kN02jUsuPlPAsPAYeas6TIPx2ogkbX1dLR0utVORf7LI389/nXXdicjo8xn6UVVRKnFzGxHX7t8Y+2ni6Tmoujr6wuFjfVEhR2bikJ9t3ROb2lpATJDfUNUgEvooh6W4bpsfIEgkov5kmOipS/6VRMUUz6jiVqcXzZ27FjUsRajsaiATjHQtB87fkyncgizjsDsXFVz2bKI4dog40U24JCS4s/ceUXc+cuuqWPdZFTEYnXMzE057DYuR3pMZRqp4Ncf4qJ2bBg3fFRgSEBba9WDpG7fz43dHGU/e7lcLgB4ExN5ne2KQEN9U5vprZ2dC5NIJBmhviEqon33jUI772Ts8lETFBvtV7ypqmcZeY9FP7A1/3PqkRa/WdtiiKtHxz93WfqrKzcv4kcGrnENF69r7+YYHjG9orbm+vVGvIvfuRufWHT2FSutIt7axhoRVrS2tIEu0dx43JqYXQdDv1iDfsObHTb7VnzSveR/l65eaKstT9ANSVpaWBoaWhY25gJB3fqIDXK+5y8LD0JDfZ+8nNlMptjZt7vxQBASoZggI9Q3RGUEH0+KjAuJC4kK736ZvjilB1ZGp4DIJLkKD1IUU356ZoZAE2fvZAcASL6WYOzv+e9P10d4jLczYJ+MufjR5s8ssVjnCa7cs0LviZKf38/+7++4G+lHTx8eZS/dK5PUig4ODkLhfwwKAzh0CsstFBL37trnEPCR6BveiLEj3L1GPMutzE7Nsw0Yr9B5obBaWrR0tC2szQBg+4aGy7O8C2+AfsDXmDljZsLFpOePXnqtnIdmNdWSc1Lzp4QtRUN9Q/qUt9rHABl7dFCSozAhcUDNda+Q8oVCYk7may63/u+4c8/06BjrsR9PNDpMapo93/vGyfhxi+ZbYrEAAA6HVUXG2DvaSlTncXn1ldnzZi30n+4fGhr6wSxfCWf6Uiv6B/k67D2W/7poqnfHtwMEIR3+/XAWUXjql9kdZ6JhN95n3JPs89ev3QgMGK+oL2gEIRUVvJkcNMnFdAgWDFm+fLlC1UdNDli5dumFf/4JmOY3aqglgpAunL2Ecx4jCvUN6WOCUX/7Kiw4uGn3ui0PtIbq0/svhn0+6+XLkjH+wetWzkOErFcpLwsrqYFzl0x2b49sUpqbnVXbHDre5tzJo09yieZDh+H1tSi1hc+fVny5Y4XzEM2C1AcXLlzIyCKMmeJrKjZpL7Wijr4Bsa6soIoS9mEgWux50rUD+w4kPs3EaptgtbV0jPHmhnr1Ja8fP7798kUKtYXXWFdeXNlMZ/McRwyTf1KdRSX9tf/Ksg3rRzjYyF1JHI2xk6a44DFX4g7dzctPOn5L22nMzt+32+m8G4eCEEgPyL+5J/3J7TFes/JJZNnFzhz54+z1+0RC5sPUAjSl/PXL9V/8LNq1JhQSr5874jdixC8HTsmuiFJVlDprUsiL/FL5TVWCG+eOrlr/Xb1A0HNRCGTgo8DcfkZGho2rnZlFtxGvADpmTivT06jIa8IFTkTX1XAuXrzgNtHdRksUVdZ67rJPQ+bPZDKZMiu2M9Rt4udrQ86cOtspdLBKoZAKEpOLt+382lLZuG4QyMBC3n90gaA241Wml7eXbG0wKfS6NgMtruCfq9ffpmnr6euVlJSIFxMKm6hE5rixY2VW7GD+imW+TnpHjl2Q01qFaGOWH/rp9LJNa1xNTXouDYEMCuRVfs2bmtI31Em+PrKLEQrKLN1sPwwL5RQVZ5TWpCcn0wFYtGghLSP9xOVEIo0LAKPmTcaBXYdcAuctCJ4iu6IYxivXfemO5xw8Fa/gCfZAK6PswB93lv3y49RxI1TbMgTyPiNndC3evwlJrQD4fPSRlczIJId/32Hl/eGCWVOeJcXvPfNw87ebpnqjiqrPfpxT3djUqmPl4ODg6uVqJm/FTrSWN+g6qTJUi4BXy9a0g5/dIOpGr+LqQSCQAQqc0IJA1BGofAhEHYHKh0DUEah8CEQdgcqHQNQRqHwIRB2ByodA1BGofAhEHYHKh0DUEah8CEQdUdjrNgB8Wm0pyrApc3zchqneKAgE0scorPzytKwSYv6eH2KFNp7nl69V4pCU2oITZ1JXbfsEU1Hw8OmD+NPX6zgcAHBTguf6T5oa/uFEBCHd/efe3cS7rworNLTtFq5YuOazj1HXNpy2qo3LNgesWLc0bJqowaqC1MtPiteuWyHbd798IKmPbsVfuQacnDlZ1Q7TApd/vkT2JqW+h7bv+99eVLXs+99vyrn0QxBS8tXke6/Ie/Z/26NnUgG39vShM6mZqYQanoWr64wZM5ct+Ui3p1qQgYcS3jzYrLLQSRPXf7tbibqvHtzctC2GxOeLUnJT7k12dfXwDCokN4mXPHt0T2BYxOuqevFELrd6efCHVxKfIgiCIHSRA5368pwd2w5VtbCVMEkM/rljMVODP86pJCIIIhDUxfy4eWnkJpE3oXcCn1+7NmLeyJEfPM8nKFpXKCQmXjm5KCTA2dl54Scb2noq38Yqj1wUPnvJ55u2R3++NMzb2dnZ2fmr7/e8y/OH9A3KKP9N5mM/5zHnbjxQvOKTTyK3E3k88UQ+vzZqSbiz8/hzN+6LEgtSHy5e8KVI9oXpj/wnz79x/xWCNP4WvbWI3EQnF0VFzPni659FVUqynn75zW+NSpxPh3mP/YYPjz1xRZRCLM8M9PLatS+uF62qAC6jKqOoTomKNemFdASpKEjxc3OTQ/ltR/f8ePTcDdHvnJS7gV5ew4f7J/6XpcTR+52kSAAAGjRXRQUHMcrM8GVmZlG1h4we76FQLTq5aP+P/1u8aqG1RqdXDBzONnR2CADU27fvoKEnSeXZu384sf77DaOGtm/Fd3CyXxzmnnDoUHxykdMwJ0xz3q6vf8NYjpsROEvUjqt3wChL3sVTV5U4IwAAAJzEO0kUnG1AkJ8oycrReoyPZ8K1228am5VtVgVoGg4d56aMX1C78SONANAz1NXS1OyxMIdTX99m9emyjigJXpNnRaxeKBAQSwdCiPnkKNSZ9luv28lRGClEJQMAgo8jhFhfBYNzDDIUVj6CkDLTMx3dnSxt3gaNojFl1mjn/OmLrfbDpk+W9MMPAPAP8nU2Ns57mVtcVM6mEf7YsX/Rlo1o/AwUPbzLum9/Op1w3meoVlVdZfob/g+nzxzd9334rAni7cyZPzvpxLncapKiJwUAYDRVPLjzBG9jZmqJFyViMNYuLs50clNeRqESbaoMAb22oq2vD1JPIE+fGyRxhxg+fAQAQFOOG8c7JjkqJA74xhLkdaLvsvGMemtfYeW30FgFWcXjJoyzxGKbqvK2Ri0fO37cwk82VLe2yqjFohHu3X4wfvx4qXNlJhbDAkMDuNyau4n//vHTXreQBaL4GW+hpj+5czTmx1NJ2SPdRjrp07d/98up+KRiEk28EN7GzMqSd/f2fUVPCgBAa6azWCwTU2Nd3U7zWXZ2dgCQs7NzlGizlxSkPdz65WdL50yfPGbq45znfX04B88JUzwlI0qzWCws1tphmENfHpn/z6ljv2zfFLViwYrI7+g84qlDvy4NCwwKX/7LvlPyuV3tNliWb6zE5EjHnaE9KNfuAwNgPNMHKBFFu5LM1B0/cVwdIfPo1dTIn484Ou85cuYFi8ECut3OAb94mFLSpLXZ27ObfL2wsLBblxPPxx1bFrl2zbI5EtnPku6v3rArIurL7VvXXNq713Ph/LVGJl+t3XL3bsCpUwdE8bA1NCzcPEfcS7oX8dliO21tgaDu4sVuw+yJY2rtOdSQyeRyLXR19CUs09MDAFCpVHnaUS0eE4NiJgbF7v42h5DmMcZNodMJme7dczk5qKqqGu03ZpyvlJGa6tCYv3oNqSJradjKmWtDTv5+K+SztavXf/7n7r3/O3rYfpj9inkzemigNDFemeiYwVtjfeOio2OSN6phuB2FlZ+elobFmwy1x5y/U/jNti8MENKx0nJTO0sjE1nf1CoqKrBYnBG+2zLDvV09vN3IWRQPD/euuf7B/pfPXPX0c8cJal+lppr7BIb4B527+nez1lCDTgW1LC0t6ytyG+rIdk72AICUxJtyhscz5FGZAOihQhdDX18fAEClUrkA9H8wbKGQWFZWbuNiZ21nCQBZ/tNRydEppIKkG2lRv+7uB2fkFYTKer6pTjN36fZI9HC+vr5HzvxTUFAAelJ+u/BDFY6O6RK6yDc6Je5m8vFgtZO+YsoXConZmTkuHi7Pz6cs/26tIQAsGrMgq3jk1FA7WUEsGUQiEYPF6Op1G3Am7VF6FVaIIPV37iTODvaXKIfBWI/3swYANNVTSgmlhJJS4D/W0tmrqy9OfT19Pp/fQGwETvY4nO3Ry//IeWoFaQ+MAWC3stsAED86u5UNADA3N+t/2QMAWmgsQl65x6wF1pqaAChwOqqg9Xr8jdnrvwgJUM3wQTapaakYDGbC3CDRXYbH4yFv77wykSX8lGhXTDT6p28s4YXk2wAq/fySUhCsbkF1FY6i/eZ16YjRQyYsnoNGzih/U9lI4/v5TZFRSyhkNZDI2no6WjrS5VOR//LIX49/3bXdycgo81l6URWxu6bMbEdfu3xj7aeLuiugr68vFDbWExvkPqd2DI0N9bS0uG3cls7pLS0tAABzcwuptfqaakI1hcafMEGZAKG95N8bVxswNp99HNoPx+Lza3PScod5OLuM7AjKTCQRATAYNqzHRaKEwhQAPIf3JN2UaFdMl/k8l+GeAKQUEpQweoCjmPIJheUkJpPVrOfj5oimZKSnc/XMx04aLV4s4cLxL7b83HEMrI6ZuSmH3cblcLu2SSMV/PpDXNSODeOGjwoMCWhrrXqQJOtt1tjNUcbjl8vlAoA3MTFW5LQAAMDCzsLcAk9tprdyOOLpJBIJALyTk1N3FfuU7OxslhbeY8zInouqlLxX91+UsjdtWN0/h6uvIFaV1XwQ+IFouaRAUPf00TOLoS7+MyYp2WjwcfGZPfQTftf5PFd3eWImD0IUG+1npGcItYZ+vGwh+lMoJGZlZDuPdrGy6TTuDgwMtBojPvjHW9tYI8KK1pY20NnPPo9bE7PrYOgXa9BveLPDZt+KT7qX/O/S1QttZb0+dEtLC0tDQ8vCxhwAIBDUrY/YIOeL8bLwqX6Bk09ezmwmU+zsrdF0BCERigmmtnZ+QROVMKaXIAgpOzvH1tXeytYCKHw6QUoftyTr2eVHJV9vWyc+YUslIUOsMUq3KZu8nIIGoblvQMcHndcvcnNTKxZ/842jgYGMiiJ83SW/SkjQHmY7JT6xdGOXLwDqONxXQPkCQV1OZo6Ll4uDy1A0hdFELc4rm/H5HLyQeHRP4rJvP0eXlRtau/pYd6rr4OAgFP7HoDCAQ0eQbKGQuHfXPoeAj0Tf8EaMHeHuNeJZbmV2ap5tgDJDXFZLi5aOtoV1+w3GNzRcnls63gAPgMbMGTMTLiY9f/TSa+U8NL2plpyTmj8lbKlyC+Z7CZvZUpRNGPXhQuu3n9MVOR25EHBrXtx9Y+Y1wd2hPbJYZcGr04mvv/p+vfioqanm9eUHhPWfzJdWhZH++CliOWaiu/3b4j2kdGmB9urVK3N7q6HO7eU5LRXHjsd5hoUt7f61ToKUQgIAsrXr6u4LQIrUrJ5fFQYfCiifWt9c+qYqcPUcC0z7vb+SUENuMRzu4Rp/+PqsL1YaAsDj1Vw5e/Xm+adfHd7r59FxG/YP8nXYeyz/ddFU7/aBK4KQDv9+OIsoPPXL7A5rNOzG+4x7kn3++rUbgQHjJabZewRBSEUFbyYHTXIxHQIAwOFsly9fLn/1UZMDVq5deuGffwKm+Y0aaokgpAtnL+Gcx2z4Zp2ChqiG6pJqGoU/btw49KeipyOCQWNyORwel9sKgMQ46lhs3J9xl4wt3Y+fPTzeeWhdSca2LQet3a3/2Ly5oxCXWZJH+f3SUalVSrKzv1q7FdiNvpBw2kFfHwDQY4pECxwO/XVGgbXLeAsdHQAAt6Xq1x9+x7sHRn+33lyJs+0WQmEK6CpydJJAlccZICig/KK8Ypa2zfgJHWEwEQRxstC9fS1p6/avnIwMAQCamvbzFs1JSigQ3b9R0LU62dnZYOU8AMDzpGvXE66/yCkbYudx7WbyJJ+JI61N60tep79Oyy/INzU1LUh98N12w8mTfD6aPU2u0R4AAAAWlVmWT167d4uy36C0P934nZvbjbO/bBU4uPILaq18Jp2+9LP1O1rB9jo3j6lnqugqaXEaSvJepqfcuZHIMzEhEur+d/jE6OHDJ06fIpo/tzS3crY0pTVWFuS8scdSvv1qRxmFWva8TKKdsQGhHtbmXauMdx5qaWfuMc5Ny8bb8O0kfI8pEi1UFVY2kprHfYA5cvKyUUtZThVm7uLowInyT224uvt2HbCXHvBzLdwhWrhTesAvJA4AEBne+fNdaUm+HK8KgxKV7wR4k/l41fpvu6ZXFaXOmhTyIr9U5UcUcePc0VXrv6sXCHou+t4jFBI3R0bMX7We2vfHSrgQ97KwrK+rdNfC+eN7R478oDetEWJ9u6zWI8RKeS/qskUHLaWWO3dUv0IjN/e1t/fYrulD3SZ+vjbkzKmz8q3HVBgKqSAxuXjbzq/7YdlJP8Ck0AszCYHTAvs6sncLlVBYp+U9UoGPF0pU6a4FBCFlpGVaOtjYOdn2XK0bXEIX+YKU+ETxaXuXjS/ad+S14xtLQCSX6pUmxqd0HQeoCaq+lVB+3PDF05w33eTS/j78e8zR86o+KNLKKNu5ZvOTjO6OO2C48ffRyA07iTze/ZvnZs1fVdPW4576XkEkBBqy+AAAEslJREFUZP0ee1bcXUJfVJHRAo1cMMPbe82WXUq3hiCIss/upEggZWW/mqBi5XM4VRGzV1W2tHRfpCXxyskDJ690X0Bh2PTS37cfKG1mqLDNd8WRPT/M/vhzYkPW+iVRj9MH/I2sJ9qunzvi7Oy8+8DJ3raEPuAV0b4aj/QRBEFUHEW7LP/5n+efHv59u+xireUNuk5dl94qiYBXy9a0ewef3foAoZD4/O5zMtZ2rPc4J8tuFzsPDlLv32eI/RwVGNgbx2fJUZiQOLH9+bIpPeDnGp0ib+lBiMqUX5WfRtNzLH5yWcvRJzzQp+cKEIiKQbUvh/jlLjiIUZnyk66eouHN6opYW9cvVUmDEAik71DxaB8CgQwIBsMHMAgEoihQ+RCIOgKVD4GoI1D5EIg6ApUPgagjUPkQiDoClQ+BqCNQ+RCIOgKVD4GoI1D54jQ8vHXx2Nmb79oMGQjodcUZz5LqOjsIhkAUReEYO72HUltw4kzqqm2fYCoKHj59EH/6eh2HAwBuSvBc/0lTwz+ciCCku//cu5t491VhhYa23cIVC9d89jG6bY3TVrVx2eaAFeuWhk0TNVhVkHr5SfHadStkRfmRCYKQ7l2/d/nS5ec5pRu3/y6ek/roVvyVa8DJmZNV7TAtcPnnS3qzn6w33Dp9Kq8y68KFu87jA0/6q2anCYKQkq8m33tF3rP/2x5dHQu4tacPnUnNTCXU8CxcXWfMmLlsyUfdBlSDvOf0867gVw9ubtoWI+7XITfl3mRXVw/PoEJyk3jJs0f3BIZFvK6qF0/kcquXB394JfEpgiAIQhe53aovz9mx7VBVC1tRe4S82tuX/loaFujs7Dxp+tyDp67WtXHfZvLPHYuZGvxxTiURQRCBoC7mx81LIzfVcTiKHkVVNJPyg7y8fthztPdNCYXExCsnF4UEODs7L/xkQ48OQNpY5ZGLwmcv+XzT9ujPl4Z5Ozs7Ozt/9f2ed3YtIL2jX5X/JvPJJ5HbiTyeeCKfXxu1JNzZefy5G/dFiQWpDxcv+FIk+8L0R/6T59+4/wpBGn+L3lpEbqKTi6Ii5nzx9c+iKiVZT7/85rdGuY3ht1ZdP3d0YbA/qvlDp68ROZ0Me5P52G/48NgTHU5EiOWZgV5eu/bFKXbaquP1y3/HDJ+S8Di1903VpBfSEaSiIMXPzU0O5bcd3fPj0XM3RL9zUu4GenkNH+6f+F9W742B9D/9955PJxft//F/i1cttNbo9IqBw9mGzg4BgHr79h00YC2pPHv3DyfWf79h1NB27x0OTvaLw9wTDh2KTy5yGuaEac7b9fVvGMtxMwJnidpx9Q4YZcm7eOpqj5bwmFXxpw4uCPt46869dQKLr77fcz0xfv2q+dZa4oZxEu8kUXC2AUF+oiQrR+sxPp4J126/aWzuxZVQnqysLIyJscdYt943ZTd+pBEAeoa6WnJ4FuZw6uvbrD5dFi5K8Zo8K2L1QoGAWFqinkGoBzz9p/zzpy+22g+bPllKPGb/IF9nY+O8l7nFReVsGuGPHfsXbdmIRt1B0cO7rPv2p9MJ532GalXVVaa/4f9w+szRfd+Hz5og3s6c+bOTTpzLrSZ1ZwOHVnnh+L75YQu3/XqwEWvz1fd7riVeWbdyXmfNAwAAo6niwZ0neBszU8uOqBUYjLWLizOd3JSXUajkVegFQiExIz3TyX2YhZHSExpKUk8gT58bJHGHGD58BABA8x25JO+e5CgMBoPpGkJP+YKDEuVm+Pgv79++du2RkZeHsCzbxG3a0k8XyfZ4y6IR7t1+MH31JqmTY6g3/r8u3r6b+G8yMdstZIEo6s5bqOlPXmSkp9cD6wluI/H69O3f/TLW29vP33eEdYdzWryNmZUl7+7t+17rVkgcgt1UFn/l2pVLVwj1DCsX780/bpu7dLaMuTpaM53FYpkMM9bV7TSHZWdnBwA5OztnYbC/jPPtC+hk6pscwozPZxsCcOnwoTeNTQAAAEzH+fqGzerbkJsOnhMcuiSyWCws1tphWNccFdBGq7hw+mImERltw0XwY/nl2XUck427v5QYMHYlOUq6t512NzwAAFFu8HGE4O7nGh0SFa6GvnmUUH7jwd/+TMokHTiyf4TFkGZS3qLg5fG3L/qPnbZn51fd1XnxMKWkSWuzd3fBTPTCwsJuXU48H3dsWeTaNcvmSGQ/S7q/esOuiKgvt29dc2nvXs+F89camXy1dsvduwGnTh0QheLQ0LBw8xxxL+lexGeLRVG9mfWEK5evxF+6Vt7MsnL13rImInyJLM2jMGhMJpdroasjEcNZT08PAEClUmVX7wsIhWXEVpOx48YAAJasX/33/iNWE2d+OMVLIKg7d+6cPC2YWnuGTFdZSOyqqqrRfmPG+UoZxPWSnBd3f9we98mOb4585QMAuHP+1KE7dyYsXN2j7FHd+8YSOgkZdbkntbzLxjOx8eqpfYWV//DWvydOPvt2/88jLIYAAHQNdDU1NMZ7TJIhewBARUUFFoszwnc7TB3u7erh7UbOonh4uHfN9Q/2v3zmqqefO05Q+yo11dwnMMQ/6NzVv5u1hnaOwKNlaWlZX5HbUEe2c7IHADCaivbG/JmU8IgGgPMYv3VfrP8ocII8bzhUCpUJgB4qdDHQcO5UKpULgIyQvn1BalqqoRXe1dMJAObNU/HjF67ytG+Pe5OSeFPOSJuqMoZCKki6kRb1626VRzfIfpa0ecPeyN0/isZ9Agyvgqn3pW+Pzh1LD+yOAyByR6eImclRqOy7cbrnsnFHZHRI3O4DW4O7BNoc1CiqfF5mRiairzvebwz6u7mewmazjXp482QQiUQMFqOr160z2bRH6VVYIYLU37mTODvYX6IcBmM93s8aANBUTykllBJKSoH/WEtnr67ue/X19Pl8fgOxETjZAwCMzEb+tC9uxYoXVy5fvnPt7qbI9AtTZy6NWPpR4ETZD3286RBjANit7DYAxI1ht7IBAObmZv0se4GgNis9Z7i7p60u9eSexFkbV4kGNTic7dHL//SvOa3X42/MXv9FSIDKRhAotIbC33+N8QyZufCjqaJjlZSUDLEy8+xxXrM0MT4F+MZu7fy83x0HgG8s4UW3ug7eGusbFx0dk7xRrR77Ct+wBUKBsfkQnXYNC27+k8AwcJg5a7qMKkIhq4FE1tbT0dKRrpeK/JdH/nr8667tTkZGmc/Si6qI3TVlZjv62uUba7sPsaqvry8UNtYTG8QTXbz8tv92KP7WxaiI4NrnyZsjVy5ZvfHmw1R+9zYbGhvqaWlx27gtndNbWloAAObmFt1X7RPI1eTKkupR40dujdpajUHslAoxrir+vXG1AWPz2cehKm/5yoWredWYFasjRPdlemPFw6Sn7l5uDj2F00aFvyjURTJNchQgiUvoIl8A4m6q10yfosrXXLYsYrg2yHiRDTikpPgzd14Rd/6ya6rE/Rhh1hGYHcfA6piZm3LYbVwOt2uLNFLBrz/ERe3YMG74qMCQgLbWqgdJj2RYYOzmKON5y+VyAcCbmBh3zbIfOWHrrn3Xk69++Ul4fdr9LVErl3yy4eaDVJ60dizsLMwt8NRmemvndbIkEgkAvJOT8oGllCM/t6CmVZfRyObz+PeTH1SyWP1sgIi8V/dflLI3bVit8pZZNELSjXujJo92eRumHQCQdPNeKZU9ZYo/pofa0oSPRsr1dQcH/DAius7lo9LPV6/vkwq/59u7OYZHTK+orbl+vRHv4nfuxieioNooZemvrty8iB8ZuMZV9PkXb21jjQgrWlvagFmn1njcmphdB0O/WIN+w5sdNvtWfNK95H+Xrl5oq9RjraWFpaGhZWHTbfxlS6fRG7ePXrFy5eVLl65evr7lv3vn/Wcui1gWMt1H/Iaio2/vFzj55OXMZjLFzt4aTUQQEqGYYGpr5xc0UQnbegE7LTXNatiwzzd8Upiol/zt/nsJj6IiwtA8gaBufcQGOd/zl4UH9caOkqxnlx+VfL1tnfgHDyoJGWLdkzDlgFxHbm6k+H/oJPpaU1+ZXcZq1NKyGuvT4zwioTAFgMjOMbJLS/IBACnRIeLze3EhGCDxzu8y3BOAuEICAOrzqq+w8s/+7++4G+lHTx8eZS89SI7zBFfuWaH3xE5d5eDgIBT+x6AwgENH4EShkLh31z6HgI9Eczkjxo5w9xrxLLcyOzXPNkCZL1WslhYtHW0LazPZxYbYua3dumvZimXxV+KvXrq6Zc2981NmRUREhMyY9PZ+ozFzxsyEi0nPH730WjkPTWqqJeek5k8JWzrMsF8j+vB4zbmZ+R6jA4bq6prN8vc6cvL6tesfLZhlq009f/zlks8m+oaGS4kc2wW8Ab7nQgAAAATcmhd335j9v71zDWrqiOL4OragKDO0KeUxFkZuEiBQhaARCT5QixKkVXn4QiE+otagErUKtdYpQqsytdhUWoZKiYO8QdomWtRqLaAIGoqJSEJ4CREkgjwKqBD6IRRDcoGboO2Yu79ve7K7s8nMuWfvnn/2zJxNsX2RNK0V30zil4d/xlbfUCkelKddlrKZ/mhDOkqu/jFg4UKjvIfW1LS0NCla+vqMjAafwN3t0uQU4ZP6+ulOdtY2VhhXjoba4Z7qnF/rQI9EwfL7GRa6iv4Svz6KIAjRcR4zLDLrYqFCq0Nvb21oEFtD3N7WLPKmUrk/ZQ9ZlEp5XHREQGiYRjW8+BOfIwjCDIscpTTfSCiV8l3M1SzOYZ3KaHcrqnnxx5cvmIUgxFUbd+ZdKh36KonfHPXyW6cSESuV8pNHDyz1H71q4CtBdrdgvr3zj+l8VTM18SSCOB6P513MTbotaxjPzNLy63QyecX67Rq1urnHDiEIQvXwK6mqU1kaKkvWLlvL0YC9dfkCf5H8EeqQyjvX6Pb29MWBtV1d2k1ti1xWutDZ2ct33e93xH9eSuewY0TSwiUuLtjk0gKWdnVMVem84VZVAT60mtv4qq2pW8xvbbinaDFOSj0lulacn59/QJDu4ukTExtFfudFZKgRVU+2fde0ozoxNbPDmPjhSj+i+VSVVkcoFIKQVQCAAkFWTl5OYZnsrWlOWecvuM+hOVoRmiTlJeW3RGIRgUAQF1+O+NR0rvuc5X5eYxzsqNHV1ikTPdoRu0+n04vJhOkbtu9fvT7ofGZealpa3YOFALgBAAAw3rw7wsEhlxe9v9+W1CdusJzjnpQaZfWfq9bKhH/1EUgz3AbVEIGhgbU1db+cFxD37qbqW3y6WXL3RknRr7n852Zmcmnjd9zEGWQybYmnKktnYW6JWBCetNSKy+7PQmyaa8oOhh+StbbJCmQa81Dn+zpZmWsPAQBYTDN3cnMwsnY1nTJFu6ltsbJz232AnZmRHnX4iNci7/BYjiQ/u77blOY+G2CjCMOGnUTxAAA1u+9Mxs9eH+gS86vLb7A/jhoK5kqlPOfsabq9fXTcGfVuyaeP8XIuyaW3rxSL1e11FcVL3RmFoqrxP65GIvdsfCg7oqlfp5A/jL6eut9+LnmJS3qtyUtJuHFP9qqHjMzjwxzWXC9/bJssAQtoV8ZFCfCoRXdRBxs22KPj03PnUhxoFOt/X8MmTLBaGbyZ4e/d2fniGH9g4GHFLZnJGzV3FRMX0YZpcmwcaFt3MJLP8LrH+7BCp/WhmH+hMvLIJ+PRlkycZOPt92qVsK8Lf7dJ7zUauTrqkMXQY8hos7U/Li0Qvu/qZKspp0KFREE5n/dZwQIAFO0JiRu0q8S9gLViWOq+SiICwINCeinrfk3A7iTGJlNMJBKJukmpVLTJO92o1CFLZ2t7Y+9Uo2f92Zk52lP4bwz2sDM5/X2K/usdgd7O6m+/SArmbCcRzMbuDRmLh1VCLu9mKCcYe35FjyGjI75TUf/4GZ3uia070TfIAxRl8DVc/weByvdJqpQeIwEATbXPYNYfZ5t9HSpqNkpKI8O+nMfcxFj2gbVZ74P7kqzUbBOix7YNfkN9bl/np12tjIkM3PrRfvapEwNSEdnHZ3huvVuQkVbVNXXXphHVOLrS0yE7dZwfwGEib2seuXcoKsJC9jX392OfzXf1ljBmwMtaG0Q/nrbXRB2Jzr4o4/K4i2c7YRpTFUcn7QEocj114T6KnE/1Md5KautaS7dJeLWsvkXRM8nS1taWNJOkkT3jfnXI0nVZwFLP64KM2OQrew9yFrjaa8/SU9082Q49KagH/c8but+chppn61BUbAnaqdOtdWu2hEPP/38pysur7+lRMxj5rlmFIY+qnwtf2DaBkTCqvtcggVW0IQaE6r+4uvg+PgM+gHfvQgwK1Wt9AgPrbRtVcSF7igBLgDe3BzDmQwyOwTs4xg7jmDsaJNDzIRA8Anf7EAgegZ4PgeAR6PkQCB6Bng+B4BHo+RAIHoGeD4HgEej5EAgegZ4PgeCRfwAk6Cql0htwiAAAAABJRU5ErkJggg=="
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "KKT条件的全称是Karush-Kuhn-Tucker条件，KKT条件是说最优值条件必须满足以下条件：\n",
    "\n",
    "![image.png](attachment:94a14d44-ffd0-4277-abb0-a53cae2c160c.png)![image.png](attachment:feeb42bc-f7fb-4eea-885d-40feb008a458.png)\n",
    "\n",
    "其中$X^*$表示样本点。这些求解条件就是KKT条件。\n",
    "\n",
    "(1)是对拉格朗日函数取极值时候带来的一个必要条件\n",
    "\n",
    "(2)是拉格朗日系数约束（同等式情况）\n",
    "\n",
    "(3)是不等式约束情况\n",
    "\n",
    "(4)是互补松弛条件\n",
    "\n",
    "(5)、(6)是原约束条件。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于一般的任意问题而言，KKT条件是使一组解成为最优解的必要条件，当原问题是凸问题的时候，KKT条件也是充分条件。\n",
    "\n",
    "对于我们的优化问题：\n",
    "\n",
    "$$\\min_w \\quad \\frac{1}{2} ||w||^2 \\\\\n",
    "s.t. \\quad y_i(w^Tx_i+b) \\geq 1 ,i=0,1,2...n $$\n",
    "\n",
    "显然，条件二已经满足了。另外两个条件为啥也满足呢？\n",
    "\n",
    "这里原谅我省略一系列证明步骤，感兴趣的可以移步这里：[点我查看](https://blog.csdn.net/xianlingmao/article/details/7919597)\n",
    "\n",
    "这里已经给出了很好的解释。现在，凸优化问题和KKT都满足了，问题转换成了对偶问题。而求解这个对偶学习问题，可以分为三个步骤：\n",
    "\n",
    "首先要让L(w,b,α)关于w和b最小化\n",
    "\n",
    "然后求对α的极大\n",
    "\n",
    "最后利用SMO算法求解对偶问题中的拉格朗日乘子。\n",
    "\n",
    "现在，我们继续推导。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**对偶问题求解**\n",
    "\n",
    "第一步：根据上述推导已知：\n",
    "\n",
    "$$ L(w,b,α) = \\frac{1}{2}||w||^2-\\sum_{i=1}^nα_i(y_i(w^Tx_i+b)-1)$$\n",
    "\n",
    "首先固定$α$，要让$L(w,b,α)$关于$w$和$b$最小化，我们分别对$w$和$b$偏导数，令其等于0，即：\n",
    "\n",
    "$$ \\frac{d L}{d w} = 0 \\implies w = \\sum_{i=1}^n α_iy_ix_i$$\n",
    "$$ \\frac{d L}{d b} = 0 \\implies  \\sum_{i=1}^n α_iy_i=0$$\n",
    "\n",
    "将上述结果带回$L(w,b,α)$得到\n",
    "\n",
    "$$L(w,b,α) = \\sum_{i=1}^n α_i-0.5*\\sum_{i,j=1}^n α_iα_jy_iy_jx_i^Tx_j$$\n",
    "\n",
    "从上面的最后一个式子，我们可以看出，此时的$L(w,b,α)$函数只含有一个变量，即$α_i$。\n",
    "\n",
    "第二步：\n",
    "\n",
    "现在内侧的最小值求解完成，我们求解外侧的最大值，从上面的式子得到\n",
    "\n",
    "$$\\max_α\\sum_{i=1}^n α_i-\\frac{1}{2}\\sum_{i,j=1}^n α_iα_jy_iy_jx_i^Tx_j \\\\\n",
    "s.t. \\,\\,\\,\\,  α_i\\geq0,i=1,2...n,  \\\\\n",
    "\\sum_{i=1}^n α_iy_i=0$$\n",
    "\n",
    "现在我们的优化问题变成了如上的形式。对于这个问题，我们有更高效的优化算法，即序列最小优化（SMO）算法。\n",
    "\n",
    "我们通过这个优化算法能得到α，再根据α，我们就可以求解出w和b，进而求得我们最初的目的：找到超平面，即\"决策平面\"。\n",
    "\n",
    "而上式可以等价于\n",
    "$$ \\min_α \\frac{1}{2}\\sum_{i,j=1}^n α_iα_jy_iy_jx_i^Tx_j  -\\sum_{i=1}^n α_i\\\\\n",
    "s.t. \\,\\,\\,\\,  α_i\\geq0,i=1,2...n,  \\\\\n",
    "\\sum_{i=1}^n α_iy_i=0$$\n",
    "\n",
    "\n",
    "总结一句话：我们为啥使出吃奶的劲儿进行推导？因为我们要将最初的原始问题，转换到可以使用SMO算法求解的问题，这是一种最流行的求解方法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**SMO算法**\n",
    "\n",
    "SMO算法的目标是求出一系列$α_i$和$b$，一旦求出了这些$α_i$，就很容易计算出权重向量w并得到分隔超平面。\n",
    "\n",
    "SMO算法的工作原理是：\n",
    "\n",
    "每次循环中选择两个$α_i$进行优化处理。一旦找到了一对合适的$α_i$，那么就增大其中一个同时减小另一个。\n",
    "\n",
    "这里所谓的\"合适\"就是指两个$α_i$必须符合以下两个条件，条件之一就是两个$α_i$必须要在间隔边界之外，而且第二个条件则是这两个$α_i$还没有进行过区间化处理或者不在边界上。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**SMO算法的解法**\n",
    "\n",
    "先来定义特征到结果的输出函数为：\n",
    "\n",
    "$$u=w^Tx+b$$\n",
    "接着，我们回忆一下原始优化问题，如下：\n",
    "\n",
    "$$\\min_w \\quad \\frac{1}{2} ||w||^2 \\\\\n",
    "s.t. \\quad y_i(w^Tx_i+b) \\geq 1 ,i=0,1,2...n $$\n",
    "\n",
    "求导得：\n",
    "\n",
    "$$w = \\sum_{i=1}^n α_iy_ix_i$$\n",
    "将上述公式带入输出函数中：\n",
    "\n",
    "$$u = \\sum_{i=1}^n α_iy_ix_i^Tx+b$$\n",
    "\n",
    "与此同时，拉格朗日对偶后得到最终的目标化函数：\n",
    "\n",
    "$$ \\min_α \\frac{1}{2}\\sum_{i,j=1}^n α_iα_jy_iy_jx_i^Tx_j  -\\sum_{i=1}^n α_i\\\\\n",
    "s.t. \\,\\,\\,\\,  α_i\\geq0,i=1,2...n,  \\\\\n",
    "\\sum_{i=1}^n α_iy_i=0$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**线性支持下的SMO**\n",
    "\n",
    "实际上，对于上述目标函数，是存在一个假设的，即数据100%线性可分。但是，目前为止，我们知道几乎所有数据都不那么\"干净\"。\n",
    "\n",
    "这时我们就可以通过引入所谓的松弛变量(slack variable)，来允许有些数据点可以处于超平面的错误的一侧。\n",
    "\n",
    "这样我们的优化目标就能保持仍然不变，但是此时我们的约束条件有所改变：\n",
    "\n",
    "$$ s.t.\\quad C\\geqα_i\\geq0,i=1,2...n,  \\\\\n",
    "\\sum_{i=1}^n α_iy_i=0$$\n",
    "\n",
    "根据KKT条件可以得出其中$αi$取值的意义为：\n",
    "\n",
    "$$α_i = 0  \\iff  y_iu_i\\geq1$$\n",
    "$$0<α_i <C \\iff  y_iu_i=1$$\n",
    "$$α_i = C  \\iff  y_iu_i\\leq1$$\n",
    "\n",
    " - 对于第1种情况，表明$αi$是正常分类，在边界内部；\n",
    " - 对于第2种情况，表明$αi$是支持向量，在边界上；\n",
    " - 对于第3种情况，表明$αi$是在两条边界之间。\n",
    "\n",
    "而最优解需要满足KKT条件，即上述3个条件都得满足，以下几种情况出现将会不满足\n",
    "\n",
    "$$y_iu_i\\leq1   \\quad  α_i <C $$\n",
    "$$y_iu_i\\geq1 \\quad  α_i > 0$$\n",
    "$$ y_iu_i=1   \\quad   α_i = 0 或者 α_i =C $$\n",
    "\n",
    "也就是说，如果存在不能满足KKT条件的$αi$，那么需要更新这些$αi$，这是第一个约束条件。此外，更新的同时还要受到第二个约束条件的限制，即：\n",
    "\n",
    "$$\\sum_{i=1}^n α_iy_i=0$$\n",
    "\n",
    "因为这个条件，我们同时更新两个$α$值，因为只有成对更新，才能保证更新之后的值仍然满足和为0的约束，假设我们选择的两个乘子为$α_1$和$α_2$：\n",
    "$$α_1^{new}y_1+α_2^{new}y_2 = α_1^{old}y_1+α_2^{old}y_2 = \\zeta$$\n",
    "\n",
    "其中， $ \\zeta$为常数。因为两个因子不好同时求解，所以可以先求第二个乘子$α_2$的解$(α_2^{new})$，得到$α_2$的解$(α_2^{new})$之后，再用$α_2$的解$(α_2^{new})$表示$α_1$的解$(α_1^{new})$。为了求解$(α_2^{new})$ ，得先确定$(α_2^{new})$的取值范围。假设它的上下边界分别为H和L，那么有：\n",
    "\n",
    "$$L\\leqα_2^{new}\\leq H$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，综合下面两个条件：\n",
    "\n",
    "\n",
    "$$ C\\geqα_i\\geq0,i=1,2...n,  \\\\\n",
    "α_1^{new}y_1+α_2^{new}y_2 = α_1^{old}y_1+α_2^{old}y_2 = \\zeta$$\n",
    "\n",
    "当y1不等于y2时，即一个为正1，一个为负1的时候，可以得到：\n",
    "\n",
    "$$α_1^{old}-α_2^{old}= \\zeta$$\n",
    "\n",
    "所以有：\n",
    "\n",
    "$$L=max(0, \\zeta),H=min(C,C-\\zeta)$$\n",
    "\n",
    "此时，取值范围如下图所示：\n",
    "\n",
    "![这里写图片描述](https://img-blog.csdnimg.cn/img_convert/13177dc74a0bb338f2c3346a1acab3af.png)\n",
    "\n",
    "当y1等于y2时，即两个都为正1或者都为负1，可以得到：\n",
    "\n",
    "$$α_1^{old}+α_2^{old}= \\zeta$$\n",
    "\n",
    "所以有：\n",
    "\n",
    "$$L=max(0, \\zeta-C),H=min(C,\\zeta)$$\n",
    "\n",
    "此时，取值范围如下图所示：\n",
    "\n",
    "![这里写图片描述](https://img-blog.csdnimg.cn/img_convert/e4266fbaf02752b9a89b70c6e8db12d7.png)\n",
    "\n",
    "如此，根据y1和y2异号或同号，可以得出$α_2^{new}$的上下界分别为：\n",
    "$$L=max(0, α_2^{old}-α_1^{old}),H=min(C,C+α_2^{old}-α_1^{old})  \\qquad if \\quad  y_1 \\neq y_2$$\n",
    "$$L=max(0, α_1^{old}+α_2^{old}-C),H=min(C,α_1^{old}+α_2^{old})  \\qquad if \\quad  y_1=y_2$$\n",
    "\n",
    "这个界限就是编程的时候需要用到的。已经确定了边界，接下来，就是推导迭代式，用于更新 $α$值。\n",
    "\n",
    "我们已经知道，更新$α$的边界，接下来就是讨论如何更新$α$值。我们依然假设选择的两个乘子为$α_1$和$α_2$。(推导略)\n",
    "\n",
    "$$ α_2^{new,clipped}= \\begin{cases} H, & α_2^{new}>H \\\\  α_2^{new}, & L\\leq α_2^{new}\\leq H  \\\\ L, & α_2^{new}<L \\end{cases} $$\n",
    "\n",
    "$$ α_1^{new}= α_1^{old}+y_1y_2(α_2^{old}-α_2^{new,clipped})  $$\n",
    "\n",
    "这样，我们就知道了怎样计算$α_1$和$α_2$了，也就是如何对选择的α进行更新。\n",
    "\n",
    "我们要根据$ α$ 的取值范围，去更正b的值，使间隔最大化。当$α_1^{new}$在0和C之间的时候，根据KKT条件可知，这个点是支持向量上的点。因此，满足下列公式：\n",
    "\n",
    "$$ y_1(w^Tx_1+b)= 1  $$\n",
    "\n",
    "公式两边同时乘以$y_1$得($y_1\\times y_1=1$)：\n",
    "\n",
    "$$  \\sum_{i=1}^nα_iy_ix_ix_1+b=y_1$$\n",
    "\n",
    "因为我们是根据$α_1$和$α_2$的值去更新b，所以单独提出i=1和i=2的时候，整理可得：\n",
    "\n",
    "$$  b_1^{new} = y_1-\\sum_{i=3}^nα_iy_ix_i^Tx_1-α_1^{new}y_1x_1^Tx_1-α_2^{new}y_2x_2^Tx_1$$\n",
    "\n",
    "其中前两项为：\n",
    "\n",
    "$$y_1-\\sum_{i=3}^nα_iy_ix_i^Tx_1=-E_1+α_1^{old}y_1x_1^Tx_1+α_2^{old}y_2x_2^Tx_1+b^{old}$$\n",
    "\n",
    "将上述两个公式，整理得：\n",
    "\n",
    "$$  b_1^{new} =b^{old}-E_1-y_1(α_1^{new}-α_1^{old})x_1^Tx_1-y_2(α_2^{new}-α_2^{old})x_2^Tx_1$$\n",
    "\n",
    "同理可得$b_2^{new}$为：\n",
    "\n",
    "$$  b_2^{new} =b^{old}-E_2-y_1(α_1^{new}-α_1^{old})x_1^Tx_2-y_2(α_2^{new}-α_2^{old})x_2^Tx_2$$\n",
    "\n",
    "当我们更新了$α_1$和$α_2$之后，需要重新计算阈值b，因为b关系到了我们$f(x)$的计算，也就关系到了误差Ei的计算。\n",
    "\n",
    "$$ b= \\begin{cases} b_1, & 0<α_1^{new}<C \\\\  b_2, & 0<α_2^{new}<C \\leq H  \\\\ (b_1+b_2)/2, & otherwise \\end{cases} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**现在，让我们梳理下SMO算法的步骤：**\n",
    "\n",
    "步骤1：计算误差：\n",
    "\n",
    "$$E_i=f(x_i)-y_i=\\sum_{j=1}^nα_jy_jx_i^Tx_j+b-y_i$$\n",
    "\n",
    "步骤2：计算上下界L和H：\n",
    "\n",
    "$$L=max(0, α_j^{old}-α_i^{old}),H=min(C,C+α_j^{old}-α_i^{old})  \\qquad if \\quad  y_i \\neq y_j$$\n",
    "$$L=max(0, α_j^{old}+α_i^{old}-C),H=min(C,α_j^{old}+α_i^{old})  \\qquad if \\quad  y_j=y_i$$\n",
    "\n",
    "步骤3：计算$η$：\n",
    "\n",
    "$$η = x_i^Tx_i+x_j^Tx_j-2x_i^Tx_j$$\n",
    "\n",
    "步骤4：更新$α_j$：\n",
    "\n",
    "$$α_j^{new}=α_j^{old}+\\frac{y_j(E_i-E_j)}{η }$$\n",
    "\n",
    "步骤5：根据取值范围修剪$α_j$\n",
    "\n",
    "$$ α_j^{new,clipped}= \\begin{cases} H, & α_j^{new}>H \\\\  α_j^{new}, & L\\leq α_j^{new}\\leq H  \\\\ L, & α_j^{new}<L \\end{cases} $$\n",
    "\n",
    "步骤6：更新$α_i$：\n",
    "\n",
    "$$ α_i^{new}= α_i^{old}+y_iy_j(α_j^{old}-α_j^{new,clipped})  $$\n",
    "\n",
    "\n",
    "步骤7：更新b1和b2：\n",
    "\n",
    "$$  b_1^{new} =b^{old}-E_i-y_i(α_i^{new}-α_i^{old})x_i^Tx_i-y_j(α_j^{new}-α_j^{old})x_j^Tx_i$$\n",
    "\n",
    "$$  b_2^{new} =b^{old}-E_j-y_i(α_i^{new}-α_i^{old})x_i^Tx_j-y_j(α_j^{new}-α_j^{old})x_j^Tx_j$$\n",
    "\n",
    "步骤8：根据b1和b2更新b：\n",
    "\n",
    "$$ b= \\begin{cases} b_1, & 0<α_1^{new}<C \\\\  b_2, & 0<α_2^{new}<C \\leq H  \\\\ (b_1+b_2)/2, & otherwise \\end{cases} $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "# python实现\n",
    "\n",
    "样本集下载："
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "!wget https://ghproxy.com/https://github.com/626626cdllp/data-mining/blob/master/SVM/testSet.txt"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "# -*- coding:UTF-8 -*-\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import random\n",
    "\n",
    "\n",
    "# 简化版smo\n",
    "\n",
    "\n",
    "# 函数说明:读取数据\n",
    "def loadDataSet(fileName):\n",
    "    alldata = np.loadtxt(fileName)\n",
    "    dataMat = alldata[:,0:2]   #添加数据\n",
    "    labelMat = alldata[:,2]   #.astype(int).reshape(-1,1)  #添加标签\n",
    "    return dataMat,labelMat\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "函数说明:随机选择alpha\n",
    "\n",
    "Parameters:\n",
    "    i - alpha_i的索引值\n",
    "    m - alpha参数个数\n",
    "Returns:\n",
    "    j - alpha_j的索引值\n",
    "\n",
    "\"\"\"\n",
    "def selectJrand(i, m):\n",
    "    j = i                                 #选择一个不等于i的j\n",
    "    while (j == i):\n",
    "        j = int(random.uniform(0, m))\n",
    "    return j\n",
    "\n",
    "\"\"\"\n",
    "函数说明:修剪alpha\n",
    "\n",
    "Parameters:\n",
    "    aj - alpha_j值\n",
    "    H - alpha上限\n",
    "    L - alpha下限\n",
    "Returns:\n",
    "    aj - alpah值\n",
    "\n",
    "\"\"\"\n",
    "def clipAlpha(aj,H,L):\n",
    "    if aj > H:\n",
    "        aj = H\n",
    "    if L > aj:\n",
    "        aj = L\n",
    "    return aj\n",
    "\n",
    "\n",
    "# 函数说明:数据可视化\n",
    "def showDataSet(dataMat, labelMat):\n",
    "\n",
    "    place_plus = np.where(labelMat==1)[0]   # 正样本的位置\n",
    "    place_minus = np.where(labelMat==-1)[0]  # 负样本的位置\n",
    "    data_plus = dataMat[place_plus]    #正样本\n",
    "    data_minus = dataMat[place_minus]  #负样本\n",
    "\n",
    "    plt.scatter(np.transpose(data_plus)[0], np.transpose(data_plus)[1])   #正样本散点图\n",
    "    plt.scatter(np.transpose(data_minus)[0], np.transpose(data_minus)[1]) #负样本散点图\n",
    "    plt.show()\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "函数说明:简化版SMO算法\n",
    "\n",
    "Parameters:\n",
    "    dataMatIn - 数据矩阵\n",
    "    classLabels - 数据标签\n",
    "    C - 松弛变量\n",
    "    toler - 容错率\n",
    "    maxIter - 最大迭代次数\n",
    "\"\"\"\n",
    "def smoSimple(dataMatIn, classLabels, C, toler, maxIter):\n",
    "    #转换为numpy的mat存储\n",
    "    dataMatrix = np.mat(dataMatIn)\n",
    "    labelMat = np.mat(classLabels).transpose()\n",
    "    #初始化b参数，统计dataMatrix的维度\n",
    "    b = 0; m,n = np.shape(dataMatrix)\n",
    "    #初始化alpha参数，设为0\n",
    "    alphas = np.mat(np.zeros((m,1)))\n",
    "    #初始化迭代次数\n",
    "    iter_num = 0\n",
    "    #最多迭代matIter次\n",
    "    while (iter_num < maxIter):\n",
    "        alphaPairsChanged = 0\n",
    "        for i in range(m):\n",
    "            #步骤1：计算误差Ei\n",
    "            fXi = float(np.multiply(alphas,labelMat).T*(dataMatrix*dataMatrix[i,:].T)) + b\n",
    "            Ei = fXi - float(labelMat[i])\n",
    "            #优化alpha，设定一定的容错率。\n",
    "            if ((labelMat[i]*Ei < -toler) and (alphas[i] < C)) or ((labelMat[i]*Ei > toler) and (alphas[i] > 0)):\n",
    "                #随机选择另一个与alpha_i成对优化的alpha_j\n",
    "                j = selectJrand(i,m)\n",
    "                #步骤1：计算误差Ej\n",
    "                fXj = float(np.multiply(alphas,labelMat).T*(dataMatrix*dataMatrix[j,:].T)) + b\n",
    "                Ej = fXj - float(labelMat[j])\n",
    "                #保存更新前的aplpha值，使用深拷贝\n",
    "                alphaIold = alphas[i].copy(); alphaJold = alphas[j].copy();\n",
    "                #步骤2：计算上下界L和H\n",
    "                if (labelMat[i] != labelMat[j]):\n",
    "                    L = max(0, alphas[j] - alphas[i])\n",
    "                    H = min(C, C + alphas[j] - alphas[i])\n",
    "                else:\n",
    "                    L = max(0, alphas[j] + alphas[i] - C)\n",
    "                    H = min(C, alphas[j] + alphas[i])\n",
    "                if L==H: print(\"L==H\"); continue\n",
    "                #步骤3：计算eta\n",
    "                eta = 2.0 * dataMatrix[i,:]*dataMatrix[j,:].T - dataMatrix[i,:]*dataMatrix[i,:].T - dataMatrix[j,:]*dataMatrix[j,:].T\n",
    "                if eta >= 0: print(\"eta>=0\"); continue\n",
    "                #步骤4：更新alpha_j\n",
    "                alphas[j] -= labelMat[j]*(Ei - Ej)/eta\n",
    "                #步骤5：修剪alpha_j\n",
    "                alphas[j] = clipAlpha(alphas[j],H,L)\n",
    "                if (abs(alphas[j] - alphaJold) < 0.00001): print(\"alpha_j变化太小\"); continue\n",
    "                #步骤6：更新alpha_i\n",
    "                alphas[i] += labelMat[j]*labelMat[i]*(alphaJold - alphas[j])\n",
    "                #步骤7：更新b_1和b_2\n",
    "                b1 = b - Ei- labelMat[i]*(alphas[i]-alphaIold)*dataMatrix[i,:]*dataMatrix[i,:].T - labelMat[j]*(alphas[j]-alphaJold)*dataMatrix[i,:]*dataMatrix[j,:].T\n",
    "                b2 = b - Ej- labelMat[i]*(alphas[i]-alphaIold)*dataMatrix[i,:]*dataMatrix[j,:].T - labelMat[j]*(alphas[j]-alphaJold)*dataMatrix[j,:]*dataMatrix[j,:].T\n",
    "                #步骤8：根据b_1和b_2更新b\n",
    "                if (0 < alphas[i]) and (C > alphas[i]): b = b1\n",
    "                elif (0 < alphas[j]) and (C > alphas[j]): b = b2\n",
    "                else: b = (b1 + b2)/2.0\n",
    "                #统计优化次数\n",
    "                alphaPairsChanged += 1\n",
    "                #打印统计信息\n",
    "                print(\"第%d次迭代 样本:%d, alpha优化次数:%d\" % (iter_num,i,alphaPairsChanged))\n",
    "        #更新迭代次数\n",
    "        if (alphaPairsChanged == 0): iter_num += 1\n",
    "        else: iter_num = 0\n",
    "        print(\"迭代次数: %d\" % iter_num)\n",
    "    return b,alphas\n",
    "\n",
    "\"\"\"\n",
    "函数说明:分类结果可视化\n",
    "\n",
    "Parameters:\n",
    "\tdataMat - 数据矩阵\n",
    "    w - 直线法向量\n",
    "    b - 直线解决\n",
    "\"\"\"\n",
    "def showClassifer(dataMat, w, b):\n",
    "    # 绘制样本点\n",
    "    place_plus = np.where(labelMat==1)[0]   # 正样本的位置\n",
    "    place_minus = np.where(labelMat==-1)[0]  # 负样本的位置\n",
    "\n",
    "    data_plus = dataMat[place_plus]    #正样本\n",
    "    data_minus = dataMat[place_minus]  #负样本\n",
    "\n",
    "    plt.scatter(np.transpose(data_plus)[0], np.transpose(data_plus)[1],s=30, alpha=0.7)   #正样本散点图\n",
    "    plt.scatter(np.transpose(data_minus)[0], np.transpose(data_minus)[1], s=30, alpha=0.7) #负样本散点图\n",
    "\n",
    "\n",
    "    #绘制直线\n",
    "    x1 = max(dataMat[:,0])  # 第一个属性的最大值\n",
    "    x2 = min(dataMat[:,0])  # 第一个属性的最小值\n",
    "    a1, a2 = w\n",
    "    b = float(b)\n",
    "    a1 = float(a1[0])\n",
    "    a2 = float(a2[0])\n",
    "    y1, y2 = (-b- a1*x1)/a2, (-b - a1*x2)/a2\n",
    "    plt.plot([x1, x2], [y1, y2])\n",
    "    #找出支持向量点\n",
    "    for i, alpha in enumerate(alphas):\n",
    "        if abs(alpha) > 0:\n",
    "            x, y = dataMat[i]\n",
    "            plt.scatter([x], [y], s=150, c='none', alpha=0.7, linewidth=1.5, edgecolor='red')\n",
    "    plt.show()\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "函数说明:计算w\n",
    "\n",
    "Parameters:\n",
    "\tdataMat - 数据矩阵\n",
    "    labelMat - 数据标签\n",
    "    alphas - alphas值\n",
    "\"\"\"\n",
    "def get_w(dataMat, labelMat, alphas):\n",
    "    alphas, dataMat, labelMat = np.array(alphas), np.array(dataMat), np.array(labelMat)\n",
    "    w = np.dot((np.tile(labelMat.reshape(1, -1).T, (1, 2)) * dataMat).T, alphas)\n",
    "    return w.tolist()\n",
    "\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    dataMat, labelMat = loadDataSet('testSet.txt')\n",
    "    showDataSet(dataMat,labelMat)\n",
    "    b,alphas = smoSimple(dataMat, labelMat, 0.6, 0.001, 40)\n",
    "    w = get_w(dataMat, labelMat, alphas)\n",
    "    showClassifer(dataMat, w, b)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# 3、非线性向量机\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "**1 核技巧**\n",
    "\n",
    "我们已经了解到，SVM如何处理线性可分的情况，而对于非线性的情况，SVM的处理方式就是选择一个核函数。\n",
    "\n",
    "简而言之：在线性不可分的情况下，SVM通过某种事先选择的非线性映射（核函数）将输入变量映到一个高维特征空间，将其变成在高维空间线性可分，在这个高维空间中构造最优分类超平面。\n",
    "\n",
    "线性可分的情况下，可知最终的超平面方程为：\n",
    "$$f(x)=\\sum_{i=1}^nα_iy_ix_i^Tx+b$$\n",
    "\n",
    "将上述公式用内积来表示：\n",
    "$$f(x)=\\sum_{i=1}^nα_iy_i<x_i,x>+b$$\n",
    "\n",
    "对于线性不可分，我们使用一个非线性映射，将数据映射到特征空间，在特征空间中使用线性学习器，分类函数变形如下：\n",
    "\n",
    "$$f(x)=\\sum_{i=1}^nα_iy_i<ϕ(x_i),ϕ(x)>+b$$\n",
    "\n",
    "其中$ϕ$从输入空间(X)到某个特征空间(F)的映射，这意味着建立非线性学习器分为两步：\n",
    "\n",
    " - 首先使用一个非线性映射将数据变换到一个特征空间F；\n",
    " - 然后在特征空间使用线性学习器分类。\n",
    "\n",
    "如果有一种方法可以在特征空间中直接计算内积$<ϕ(x_i),ϕ(x)>$，就像在原始输入点的函数中一样，就有可能将两个步骤融合到一起建立一个分线性的学习器，这样直接计算的方法称为核函数方法。"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "这里直接给出一个定义：核是一个函数k，对所有$x,z∈X$，满足$k(x,z)=<ϕ(x_i),ϕ(x)>$，这里$ϕ(·)$是从原始输入空间X到内积空间F的映射。\n",
    "\n",
    "简而言之：如果不是用核技术，就会先计算线性映$ϕ(x_1)和ϕ(x_2)$，然后计算这它们的内积，使用了核技术之后，先把$ϕ(x_1)$和$ϕ(x_2)$的一般表达式$<ϕ(x_1),ϕ(x_2)>=k(<ϕ(x_1),ϕ(x_2) >)$计算出来，这里的$<·，·>$表示内积，$k(·，·)$就是对应的核函数，这个表达式往往非常简单，所以计算非常方便。\n",
    "\n",
    "这种将内积替换成核函数的方式被称为核技巧(kernel trick)。"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "**核函数的数学要求** \n",
    "\n",
    "核函数有严格的数学要求，所以设计一个核函数是很困难的。\n",
    "\n",
    "K(x,z)是正定核的充要条件是：K（x,z）对应的Gram矩阵实半正定矩阵。 \n",
    "\n",
    "Gram矩阵：矩阵对应点的内积。KTK, KKT \n",
    "\n",
    "半正定矩阵：设A是实对称矩阵。如果对任意的实非零列矩阵X有XTAX≥0，就称A为半正定矩阵。 \n",
    "\n",
    "当检验一个K是否为正定核函数，要对任意有限输入集{xi…}验证K对应的Gram矩阵实是否为半正定矩阵。 "
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "**LIBSVM中提供的核函数** \n",
    "\n",
    "**线性核函数：**\n",
    "$$K(x,z)=x\\cdot z$$\n",
    "线性核，主要用于线性可分的情况，我们可以看到特征空间到输入空间的维度是一样的，其参数少速度快，对于线性可分数据，其分类效果很理想，因此我们通常首先尝试用线性核函数来做分类，看看效果如何，如果不行再换别的\n",
    "\n",
    "**多项式核函数：**\n",
    "$$K(x,z)=(x\\cdot z+1)^p$$\n",
    "对应的支持向量机为p次多项式分类器。\n",
    "多项式核函数可以实现将低维的输入空间映射到高纬的特征空间，但是多项式核函数的参数多，当多项式的阶数比较高的时候，核矩阵的元素值将趋于无穷大或者无穷小，计算复杂度会大到无法计算。\n",
    "\n",
    "**RBF核函数（高斯核函数） ：**\n",
    "$$K(x,z)=exp(-\\frac{||x-z||^2}{2\\sigma^2})$$\n",
    "对应的支持向量机为高斯径向基函数分类器。\n",
    "高斯径向基函数是一种局部性强的核函数，其可以将一个样本映射到一个更高维的空间内，该核函数是应用最广的一个，无论大样本还是小样本都有比较好的性能，而且其相对于多项式核函数参数要少，因此大多数情况下在不知道用什么核函数的时候，优先使用高斯核函数。\n",
    "\n",
    "**sigmoid核函数：**\n",
    "$$\\kappa(x,z) = tanh(\\eta<x,z> + \\theta)$$\n",
    "采用sigmoid核函数，支持向量机实现的就是一种多层神经网络。"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "SVM的核函数如何选取\n",
    "-----------\n",
    "最常用的是Linear核与RBF核。需要注意的是需要对数据归一化处理。\n",
    "\n",
    "1、Linear核：主要用于线性可分的情形。参数少，速度快，对于一般数据，分类效果已经很理想了。\n",
    "\n",
    "2、RBF核：主要用于线性不可分的情形。参数多，分类结果非常依赖于参数。\n",
    "\n",
    "有很多人是通过训练数据的交叉验证来寻找合适的参数，不过这个过程比较耗时。\n",
    "\n",
    "我个人的体会是：使用libsvm，默认参数，RBF核比Linear核效果稍差。\n",
    "\n",
    "通过进行大量参数的尝试，一般能找到比linear核更好的效果。\n",
    "\n",
    "如果特征的提取的好，包含的信息量足够大，很多问题都是线性可分的。\n",
    "\n",
    "当然，如果有足够的时间去寻找RBF核参数，应该能达到更好的效果。\n",
    "\n",
    "这些函数中应用最广的应该就是RBF核了，无论是小样本还是大样本，高维还是低维等情况，RBF核函数均适用\n",
    "\n",
    "它相比其他的函数有一下优点：\n",
    "\n",
    "1）RBF核函数可以将一个样本映射到一个更高维的空间，而且线性核函数是RBF的一个特例，也就是说如果考虑使用RBF，那么就没有必要考虑线性核函数了。\n",
    "\n",
    "2）与多项式核函数相比，RBF需要确定的参数要少，核函数参数的多少直接影响函数的复杂程度。\n",
    "\n",
    "另外，当多项式的阶数比较高时，核矩阵的元素值将趋于无穷大或无穷小，而RBF则在上，会减少数值的计算困难。\n",
    "\n",
    "3）对于某些参数，RBF和sigmoid具有相似的性能。\n",
    "\n",
    "因此，在选用核函数的时候，如果我们对我们的数据有一定的先验知识\n",
    "\n",
    "就利用先验来选择符合数据分布的核函数；\n",
    "\n",
    "如果不知道的话，通常使用交叉验证的方法，来试用不同的核函数，误差最下的即为效果最好的核函数，或者也可以将多个核函数结合起来，形成混合核函数。\n",
    "\n",
    "在吴恩达的课上，也曾经给出过一系列的选择核函数的方法：\n",
    "\n",
    "下面是吴恩达的见解：\n",
    "\n",
    "1、如果Feature的数量很大，跟样本数量差不多，这时候选用LR或者是Linear Kernel的SVM\n",
    "\n",
    "2、 如果Feature的数量比较小，样本数量一般，不算大也不算小，选用SVM+Gaussian Kernel\n",
    "\n",
    "3、 如果Feature的数量比较小，而样本数量很多，需要手工添加一些feature变成第一种情况"
   ],
   "metadata": {
    "collapsed": false
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  },
  "pycharm": {
   "stem_cell": {
    "cell_type": "raw",
    "source": [],
    "metadata": {
     "collapsed": false
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}